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PREFACE 


The  work  described  herein  was  carried  out  over  the  period  9 February 
1972  through  30  June  1975.  This  report  summarizes  the  work  of  many  people 
on  a complex  topic.  Most  of  the  experiments  and  results  have  been  detailed 
in  the  six  semi-annual  technical  progress  reports.  This  technical  report 
describes  only  those  relevant  experiments  that  led  to  the  conclusions  dis- 
cussed on  page  77. 

Various  portions  of  this  research  program  have  been  presented  orally 
in  several  scientific  forums.  The  participants  made  critical  and  construc- 
tive comments  that,  in  most  cases,  were  later  incorporated  into  the  proj- 
ect. The  more  important  of  these  forums  were  the  following: 

• The  San  Francisco  Bay  Area  Neurolinguistics  Society  meeting, 
Stanford  University,  Dr.  Karl  Pribram,  Chairman. 

• The  Department  of  Linguistics,  University  of  California, 

Berkeley,  sponsored  by  Dr.  William  Wong. 

• The  Slectroencephalographic  Laboratory  of  the  Langley  Porter 
Institute  of  Psychiatry,  University  of  California  Medical 
Center,  San  Francisco,  under  the  direction  of  Dr.  Enoch 
Calloway. 

• The  Brain  and  Language  Symposium  at  the  Brain  Research  In- 
stitute, University  of  California  at  Los  Angeles,  under  the 
sponsorship  of  Dr.  Donald  Walters. 

The  most  useful  and  critical  forums,  however,  were  the  several  Contractor's 
Meetings  of  the  ARPA-sponsored  Biocybernetic  Program,  from  which  much  val- 
uable advice  was  received. 
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SUMMARY 


The  purpose  of  this  three-year  research  program  was  to  test  the 
feasi  lility  of  designing  a close- coupled,  two-way  communication  link 
between  man  and  computer  using  biological  information  from  muscles  of 
the  vocal  apparatus  and  the  electrical  activity  of  the  brain  during 
overt  and  covert  (verbal  thinking)  speech. 

The  research  plan  was  predicated  on  existing  evidence  that  verbal 
ideas  or  thoughts  are  subvocally  represented  in  the  muscles  of  the  vocal 
apparatus.  If  the  patterns  of  this  muscle  activity  are  at  all  similar 
to  those  involved  in  normal  overt  speech,  a reasonable  assumption  is 
‘bat  the  electrical  activity  of  the  brain  during  verbal  thinking  may  be 
similar  to  that  during  overt  speech. 

During  the  first  two  years,  we  simultaneously  recorded  electro- 
myographic signals  from  the  facial  muscles  involved  in  speech  and  the 
correlated  electroencephalographic  (EEG)  signals  from  overlying  areas 
of  the  cerebral  cortex  involved  in  speech.  Ve  digitized  the  analog  out- 
put for  computer  processing  and,  using  statistics  designed  to  reveal 
patterns  of  cortical  activity,  formulated  a computer  pattern-recognition 
program  to  identify  features  in  the  physiological  data  that  were  associ- 
ated with  specific  words,  whether  overtly  or  covertly  produced. 

During  the  third  year,  we  concentrated  on  improving  predictive 
power,  by  searching  and  eliminating  sources  of  error,  and  on  devising 
computer  programs  for  real-time  analysis  of  the  EEG,  as  would  be  used 
in  an  on-line  biocybernetic  communications  system.  In  addition,  extensive 
investigations  were  conducted  on  the  role  of  cerebral  hemispheric  EEG 
asymmetry  related  to  language  and  nonlanguage  tasks  and  to  performance. 

The  results  are  reported  in  two  parts.  Part  I concerns  the  off-line 
and  on-line  analysis  of  the  EEG  coincident  with  overt  and  covert  speech 
as  it  might  be  used  in  biocybernetic  communication,  and  Part  II  concerns 
the  hemispheric  laterality  difference. 
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Part  I--Biocybern -tic  Communication 


Off-Line  Analysis,  Overt  Classification 

We  conducted  a computer  analysis  of  EMG  and  EEG  recordings  from  each 
of  three  subjects  during  performance  of  a language  task  on  two  separate 
occasions  so  as  to  determine  whether  the  computer  could  correctly  clas- 
sify 15  overtly  spoken  Engli'-1’  words  based  on  these  electrophysiologi- 
cal  patterns  alone.  Several  statistics  later  were  applied  to  the  EMG 
and  EEG  responses  that  were  coincident  with  the  15  word  utterances  (each 
repeated  ten  times  at  each  of  the  two  sessions),  but  only  one  statistic 
was  found  useful  for  successful  pattern  recognition.  This  was  based  on 
calculating  an  average  response  for  each  electrode  for  the  period  three 
seconds  before  and  three  seconds  after  the  onset  of  vocalization  of  a 
word.  Each  of  the  15  average  responses  per  electrode  (six  electrodes,  or 
90  average  responses  for  the  15  words  per  subject  per  session)  then  served 
as  a template  against  which  individual  responses  were  compared.  These 
comparisons  were  made  by  calculating  the  RMS  (root-mean-squara)  dif- 
ference between  a single  response  of  each  electrode  and  the  15  word 
templates  for  that  electrode.  The  individual  electrode  response  was 
then  classified  as  th,  word  for  that  template  with  which  the  RMS  dif- 
I ference  was  a minimum. 

The  significant  results  were: 

(1)  Both  EMG  and  EEG  responses,  taken  separately  or  together, 
were  used  to  classify  any  one  or  all  of  the  15  overtly 
spoken  words.  The  percentage  of  correct  classifications 
for  all  electrodes  of  the  three  subjects  for  two  sessions 
each  ranged  from  9 to  84%. 

(2)  Out  of  5,400  possible  correct  classifications  across  all 
subjects  and  sessions,  74%  of  the  words  were  classified 
correctly  by  EMG  responses  alone,  63%  by  EMG  plus  all  EEG 
responses,  and  34%,  by  EEG  responses  alone.  Chi  square 
tests  of  significance  showed  that  these  correct  classifi- 
cations would  have  occurred  by  chance  with  a probability 
of  less  than  1 in  1,000. 

| • 

(3)  Reliability  within  each  subject  from  one  session  to  the 
other  was  high.  Templates  for  one  session  of  a given  sub- 
ject could  serve  to  classify  words  correctly  based  on  EMG 
and  EEG  responses  of  the  other  session  nearly  as  well  as 
templates  within  a session.  In  addition,  the  rate  of 
correct  classifications  was  higher  for  the  second  session 
than  for  the  first  for  all  three  subjects,  indicating  a 
learning  or  habituation  effect  that  lowered  response 
variability. 


( ) When  templates  of  one  subject  were  used  to  classify  words 
based  on  individual  responses  of  another  subject,  the 
percentage  of  correct  classifications  for  EEG  responses 
was  no  greater  than  chance  expectation.  The  percentage 
of  correct  classifications  for  EMG  was  greater  than  chance, 
ut  not  nearlv  So  good  as  within  subjects.  Thus,  each 
subject's  biological  patterr  associated  with  speech 
appear  to  be  unique. 

(5)  Six  possible  sources  of  error  in  word  classification  were 
identified,  and  their  relative  contribution  to  decreased 
success  was  evaluated.  It  was  determined  that,  if  all 
sources  of  error  could  be  eliminated,  significant  gains 
in  correct  word  classification  using  biological  responses 
would  be  achieved  (perhaps  approaching  90%  or  better). 

We  conclude  from  these  results  that  it  is  feasible  for  a human  to 
communicate  verbally  overtly  with  a computer,  using  biological  informa- 
t on  alone,  with  a high  degree  of  accuracy  and  reliability,  at  least 
under  conditions  of  a limited  vocabulary. 
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Off-Line  Analysis.  Covert  Classification 

This  portion  of  the  report  describes  attempts  to  determine  whether 
EG  responses  associated  with  covert  speech  resemble  those  associated 
with  overt  speech  of  the  same  words  and  whether  they  also  can  be  classi- 
ied  by  the  computer.  Subjects  were  two  right-handed  female  volunteers. 
Covert  speech  was  defined  as  the  silent  reading  of  words  visually  pro- 
jected on  a rear-vision  screen.  Five  words  were  used  for  each  subject 
and  covert  speech  results  were  compared  with  overt  speech  results  for 
the  same  subjects  on  the  same  words  during  the  same  session. 


EEG  responses  for  covert  speech  mimicked  those  of  overt  speech  for 
the  same  subject,  electrode,  and  spoken  word.  When  sources  of  error  were 
reduced  as  much  as  possible,  correct  computer  classification  rates  ranged 
rom  52  to  727,,  which  was  significant  at  p < 0.001.  We  conclude  that 
both  overt  and  covert  speech  can  be  identified  by  computer  classifica- 
tion of  electrophysio  logical  responses  and  that  a practical  biocybernetic 
communication  system  is  feasible,  providing  the  sources  of  error  can 
be  removed. 


Articulatory.  Semantic,  and  Contextual  Component? 


These  experiments  were  designed  to  determine  whether  semantic  and 
motor  components  are  in  the  EEG  response  and  are  associated  with  overt 
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speech.  We  compared  computer  classification  of  the  EEC  and  EMC  during 
speaking  of  similarly  pronounced  words  and  during  speaking  of  homon^. 

When  spoken  in  isolation,  both  similarly  pronounced  words  and 
homonyms  were  classified  poorly  by  the  computer  unless  definite  mechan- 
ical (arLiculatory)  differences  existed.  However,  the  percentages  of 
correct  responses  improved  significantly,  especially  in  the  EEG^  when 
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On-Line  Analysis 


The  results  from  the  off-line  computer  analysis  of  the  EEC  coinci- 
dent with  overt  and  covert  speech  were  sufficiently  above  chance  expecta- 
tion for  us  to  conclude  that  a biocybernetic  communication  system  as 
originally  proposed,  is  feasible.  However,  to  be  useful,  such  a communi- 
cation  system  must  be  employable  In  real-time,  with  direct  on-line  com- 
munication between  a man's  brain  and  a computer.  The  last  section  of 
Hart  I describes  our  results  toward  this  end 


session^  °J  31  SUbjeCtS  WGre  run  for  a total  of  140  experimental 
sessions.  Subjects  were  both  male  and  female  volunteers,  ranging  in  ave 

pariso  ^ ‘ Data  analysis  included  normalization  of  data,  com- 

parison of  machine  versus  human  time  justification,  template  updating 

using  an  exponential  decay  method,  and  comparison  of  the  Mean  Square 

wideevaerietmetf  ^ ^ Cl3Ssif ication  with  « correlation  method.  A 

repetitive!  Were  inVes  tlgated,  including  comparison  of 

petitively  presented  stimulus  words  versus  random  presentation,  overt 
versus  covert  speech,  left  versus  right  versus  both  cerebral  hemispheres 
ramed  versus  nontrained  subjects,  and  "best  word"  percentage  of  correct’ 
lassif ications  versus  averages  over  all  words. 


The  significant  results  were: 


(1) 


On-line  classification  techniques  usi>  > the  Linc-8  computer 
provided  EEC  classification  equal  to  or  better  than  clas- 

si  ication  using  the  original  off-line  (CDC  6400)  analysis 
techniques . 


(2) 


Covert  responses  generally  were  classified  correctly  less 
often  than  overt  responses,  although  the  reverse  was  true 

m all  cases  for  one  subject  and  in  two  cases  for  another 
subject. 
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(3)  In  about  half  the  subjects,  the  right  hemisphere  tended  to 
be  equal  or  superior  to  the  left  hemisphere  and  both  hemi- 
spheres taken  together  in  the  classification  of  EEG  re- 
sponses to  both  overt  and  covert  speech.  For  the  other 
half,  the  left  hemisphere  tended  to  be  superior. 

(4)  Practice  was  a vital  element  in  improving  classification, 
particularly  for  covert  EEG  responses.  The  data  clearly 
indicated  that  words  are  classified  correctly  more  often 
after  the  first  few  repetitions  (i.e.,  practice  responses). 

(5)  Single  "best  words"  usually  were  classified  correctly  a 
high  percentage  of  the  time  (70  to  1007o),  indicating  the 
strong  feasibility  of  improving  overall  correct  classifi- 
cation rates. 

(6)  The  "over  all  words"  percentage  of  correct  classification 
rates  were  lower  than  those  for  "best  words." 

(7)  EEGs  associated  with  repetitively  produced  words  were  clas- 
sified correctly  significantly  more  often  by  on-line  com- 
puter analysis  than  those  for  randomly  produced  words, 
sugges ting  that  anticipation  improves  the  consistency 

of  the  EEG  associated  with  a particular  word.  This  may 
be  related  to  level  or  strength  of  cortical  organization, 
in  which  good  cortical  organization  for  language  may 
depend  on  preparation  or  set. 

(8)  Trained  subjects  generally  had  a higher  percentage  of  cor- 
rect classifications  than  nontrained  subjects. 


(9)  The  Mean  Square  Difference  method  of  machine  classification 
was  slightly  superior  to  the  correlation  method,  although 
results  were  inconsistent  across  the  various  parameters. 

(10)  Exponential  updating  of  templates  was  superior  to  simple 
averaging  in  machine  classification. 

(11)  Human  visual  time  justification  was  superior  to  machine 
time  justification,  significantly  improving  the  percent- 
age of  correct  classifications  (up  to  60%). 
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Conclusions 


Man-machine  interaction  using  the  EEG  coincident  with  overt  or  co- 
vert speech  is  feasible  as  a biocybernetic  communication  system.  However, 
further  research,  especially  in  the  areas  of  EEG  variability,  covert 
speech,  and  pattern  recognition  techniques,  must  be  conducted  before  a 
practical  system  is  realizable.  From  our  observations,  optimum  parameters 
may  be  specified  for  an  accurate  bra in- computer  intercommunication  system 
as  follows: 

(1)  Subjects  who  have  strong  hemispheric  lateralization  for 
language  should  be  chosen.  This  may  be  predetermined  by 
dichotic  listening  tests  (see  Part  II). 

(2)  As  many  as  six  to  eight  electrodes  should  be  employed 

on  the  side  of  cerebral  dominance  for  language,  with  em- 
phasis on  cortical  areas  involved  in  speech-,  word-,  and 
auditory-association  areas. 

(3)  Subjects  should  be  trained  in  the  operating  system, 
especially  temporal  training  for  covert  responses,  and  to 
speak  or  think  words  as  consistently  as  possible. 

(4)  Subjects  should  know  what  words  will  be  said  or  thought 
next;  i.e.,  they  should  have  a "set"  to  respond. 

(5)  Words  for  speaking  or  thinking  should  have  some  meaning 
for  the  subject  or  should  be  used  in  a meaningful  context. 

(6)  Data  collected  from  each  EEG  electrode  should  be  limited 
as  nearly  as  possible  to  the  time  required  for  actual 
utterance  or  thought. 

(7)  Data  from  all  EEGs  should  be  strung  together  in-line,  as 
a single  response,  and  should  be  smoothed  sufficiently 
to  eliminate  high-frequency  components. 

(8)  Pattern- recognition  algorithms  the  computer  employs  should 
time- justify  the  single,  combined  response,  much  as  a human 
observer  would  do  with  visual  pattern  recognition. 

(9)  Stored  pa tterns  should  be  "refreshed"  periodically  (i.e., 
new  templates  should  be  formed  and  updated)  to  take  ac- 
count of  "drifting"  cortical  organization. 


6 


Part  II--Hemispheric  Laterality 

No  a Rriori  evidence  exists  that  cortical  organization  for  language 
production  or  verbal  thinking  resides  only  in  the  sites  we  have  chosen  to 
place  our  EEG  electrodes.  Indeed,  the  evidence  we  and  others  have  ob- 
tained shows  that,  in  some  people,  cortical  organization  for  speech 
occurs  in  several  locations  other  than  those  investigated.  Accordingly, 
three  studies  were  carried  out  (two  in  depth)  to  determine  the  effects  ’ 
of  laterality  in  language  and  nonlanguage  tasks  and  the  relationships  be- 
tween correct  computer  classification  of  speech  and  performance. 


Dichotic  Correlations  1 

i 

Dichotic  listening  denotes  the  simultaneous,  bilateral  presentation 
of  two  different  auditory  stimuli  to  determine  performance  differences 
between  the  two  ears  and,  consequently,  to  demonstrate  lateralization 
of  function. 

Eight  right-handed  subjects  (who  were  presumably  left  hemisphere 
dominant  for  speech)  were  given  dichotic  listening  tests  for  specifica- 
tion of  hemispheric  language  dominance.  The  dichotic  scores  showed  that 
four  of  the  subjects  were  left  hemisphere  dominant,  three  were  right 
hemisphere  dominant,  and  one  was  bilateral.  These  scores  were  compared 
with  percentages  of  correct  computer  classifications  of  EEG  responses 
recorded  from  the  right  and  left  hemisphere,  resulting  in  a positive 
and  significant  correlation  (Rho  = 0.78,  p < 0.05).  This  suggests  that 
handedness  cannot  be  employed  as  a referent  for  electrode  placements  for 
biocybernetic  communication,  that  percentages  of  correct  EEG  response 
classifications  may  be  improved  by  giving  the  dichotic  listening  test  be- 
fore measurement,  and  that  different  EEG  sites  will  be  required  for  bio- 
cybernetic communication  with  different  people. 

EEG  Asymmetry  and  Word  Classification 

The  studies  on  biocybernetic  communication  showed  that  correct  com- 
puter classification  of  EEG  responses  and  hemispheric  lateral  dominance 
for  language  perception  (dichotic  listening)  are  positively  and  signifi- 
cantly correlated.  In  this  study,  subjects  were  chosen  on  the  basis  of 
known  laterality  differences  (six  stutterers,  eight  nonstutterers,  all 
right-handed)  found  in  language  and  music  tasks,  which  then  were  compared 
for  computer  EEG  classification  (off-line)  during  overt  speech. 

Two  measures  were  used  to  assess  laterality  of  speech  in  right- 
handed  stutterers  and  nonstutterers:  (1)  dichotic  listening  and 
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(2)  asymmetrical  changes  in  EEG  alpha  (8  to  12  Hz)  during  two  vocal 
tasks--speaking  and  singing.  Verbal  to  music  ratios  (V/M)  of  average 
alpha  amplitudes  were  computed  for  each  of  four  temporal  EFJ  electrodes: 

T3  T4  T5,  and  T6.  V/M  ratios  at  both  left  hemisphere  sites  were 
significantly  lower  (p  < 0.05)  than  at  right  sites  in  right- eared  non- 
stutterers but  not  in  nonright- eared  nonstutterers  or  in  stutterers. 

Among  nonstutterers,  there  was  a significant  positive  rank  correlation 
between  dichotic  scores  and  T3  V/M  ratios  (rs  = 0.81,  p < 0.01):  the 

greater  the  right  ear  preference,  the  lower  the  T3  V/M  ratio.  The  same 
correlation  in  stutterers  was  slightly  negative  and  nonsignificant. 

When  computer  classification  for  the  EEG  during  overt  speech  for 
these  groups  was  compared,  the  results  showed1 

(1)  The  percentage  of  correct  classification  in  known  speech 
areas  (temporal  leads)  was  greater  than  chance  (often 

p < 0.05),  whereas  in  nonspeech  areas  (parietal  leads), 
the  percentage  of  correct  classification  was  no  better 
than  chance. 

(2)  No  signifnicant  differences  were  found  in  percentage  of 
correct  classifications  between  groups. 

(3)  No  significant  correlations  were  found  among  or  between 
groups  between  hemispheres  and  percentage  of  correct 
classifications . 

(4)  There  was  a significant  positive  correlation  between  per- 
centage of  correct  classifications  and  strong  ear  prefer- 
ence, indicating  the  greater  the  cortical  organization 
for  language,  the  more  the  EEG  was  consistently  associated 
with  a particular  word. 

(5)  A significant  and  positive  correlation  was  found  between 
the  percentage  of  correct  classification  of  the  EEG  by 
computer  during  overt  speech  and  the  amount  of  laterality 
difference  on  language  and  nonlanguage  tasks. 

We  conclude  from  these  findings,  and  the  results  from  repetitive 
data  obtained  in  on-line  analysis,  that  cortical  organization  in  subject- 
dependent  (i.e.,  some  subjects  have  a more  restricted  cortical  organiza- 
tion for  language  than  others),  and  that  percentage  of  correct  computer 
classification  rates  are  a function  of  this  dependence  as  well  as  of  a 
preparatory  set  to  respond. 

The  second  hemispheric  laterality  study  was  concerned  with  identify- 
ing asymmetrical  function  of  the  two  cerebral  hemispheres  during  language 
and  nonlanguage  tasks  and  relating  any  differences  to  performance. 
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13  w.r.  I J6CtS  were  given  various  testa  for  laterality,  and 

13  were  gtven  preliminary  EEG  evaluations.  Computer-controlled  tasks 
were  designed  to  present  either  words  or  spatial  patterns  rapidly  to 

ter  SU  rCt'a  ^ sub^ecc's  task  was  to  discriminate  between  the  pat- 
terns and  words.  EEG  laterality  measures  then  were  correlated  with  the 
subject's  reaction  time  and  discriminative  performance. 


TargPt  Recognition  and  EEG  Aloha  Asymmetry 
General  Task  Development 


An  experimental  paradigm  was  conceived  and  developed  that  would 
. 63  Presum^bly  to  differential  engagement  of  right  and  left  hemispheres 

reaction  “°Uld  “1'"  °f  “r8et  deteati°a  Performance 

reaction  time,  EEG  power  spectrums,  and  visually  evoked  potentials  to 

target  and  nontarget  words  and  pattern  stimuli. 


Computer  Program  Development 

Exper  mental  Control--Several  programs  were  written  for  the 
inc-8  to  allow  automatic  presentation  of  verbal  or  spatial  stimuli  with 
control  of  several  stimulus  parameters  such  as  stimulus  duration,  fre- 
quency, number  of  targets,  number  of  task  repetitions,  and  so  on.  Be- 
havioral responses  were  categorized  by  the  computer  in  terms  of  target 
an  nontarget  hits  and  misses,  and  reaction  times  for  each  category  were 
obtained.  Modification  of  the  programs  allowed  on-line  triggering  of 
stimuli  as  a function  of  hemispheric  alpha  asymmetry.  Either  words  or 
patterns  alone  or  a mixture  of  words  and  patterns  could  be  present. 


Data  Analysis --Programs  were  written  to  facilitate  spectral 
ana  ysis  of  EEGs  and  to  obtain  quantification  of  spectral  parameters. 
One  program,  COOLEY,  performed  spectral  analysis  of  sequential  blocks 

AVETYPE^obt  i^d  “2  3 ^ °n  Linc  tape.  Another  program, 

ETYPE  obtained  and  printed  several  parameters  of  the  spectra  for  any 

selected  band,  including  average  power,  peak  power,  peak  position  (fre- 
quency), frequency  centroid,  and  measures  of  alpha  dispersion. 


Experimental  Results 

. t , MG  Asymmetry  and  Target  Recognition- Ei vhi-  subjects  screened 
for  handedness  and  normal  EEGs  were  tested  in  verbal  or  pattern  target 
recognition.  Right-hemisphere  alpha  was  augmented  in  the  spatial  task 


contrary  to  expectations,  but  reaction  time  to  the  stimulus  was  fastest 
when  the  hemisphere  appropriate  to  a stimulus  category  (verbal  or  spa- 
tial) was  most  aroused,  as  indicated  by  EEG  asymmetry;  e.g.,  reaction 
tine  to  words  was  fastest  when  the  left  hemisphere  was  most  aroused  rela- 
tive to  the  right. 

Evoked  potentials  to  words  were  indistinct  but  were  clear  in 
response  to  patterns.  The  major  component  of  the  spatial  EP  to  target 
stimuli  was  larger  in  the  right  than  in  the  left  hemisphere  in  seven  of 
the  eight  subjects. 


On-Line  Trigger  of  Stimuli  by  EEC  /*symmetry--Because  alpha 
activity  apparently  reflects  cortical  "idling,"  it  was  predicted  that 
triggering  a word  stimulus  with  a relatively  larger  left-hemisphere  alpha 
burst  would  result  in  slower  reaction  times  to  words  than  when  they  were 
triggered  by  right-hemisphere  alpha  increments.  Of  11  subjects,  four 
behaved  as  predicted,  five  behaved  opposite  the  prediction,  and  two 
showed  no  consistent  verbal-spatial  relationship.  Thus,  in  9 of  11  sub- 
jects, the  hemisphere  responding  most  efficiently  to  patterns  was  the 
opposite  of  that  responding  most  efficiently  to  words. 


Volitional  Trigger  of  Words  and  Patterns:  Biof eedback--Obser- 

vations  were  made  in  one  subject  under  conditions  where  right-hemisphere 
alpha  activicy  always  triggered  words  and  left-hemisphere  alpha  triggered 
patterns.  The  subject  learned  rather  easily  to  increase  the  production 
of  words--i.e.,  to  augment  the  normal  high  incidence  of  right-hemisphere 
alpha--but  only  with  difficulty  was  she  able  to  increase  relative  left- 
hemisphere  alpha  and  reduc  ’ it  in  the  right  hemisphere. 

On-Line  Trigger  of  Mixed  Words  and  Patterns:  Pattern  Diffi- 

cul ty--The  computer  program  was  modified  for  evaluating  an  on-line  para- 
digm where  either  left  or  right  alpha  increments  triggered  words  or  pat- 
terns randomly.  Preliminary  runs  with  two  subjects  suggested  that  pattern 
difficulty  might  be  an  important  parameter  influencing  EEG-performance 
relationships.  Six  additional  subjects  were  screened  for  handedness, 
and  four  were  tested  in  this  paradigm  with  variation  of  pattern  dif- 
ficulty. The  data  were  generally  unsatisfactory  because  of  lack  of  alpha 
in  two  cases  and  excessive  eye  blinks  and  muscle  artifacts  in  the  other 
two.  Pursuing  this  exact  methodology  further  did  not  seem  reasonable 
because  of  the  technical  difficulties.  Mixing  the  verbal  and  nonverbal 
tasks  seems  to  induce  excessive  stress  and  consequent  artifacts.  However, 
the  use  of  alpha  frequency  in  this  paradigm  might  produce  clearer  results, 
as  it  should  be  less  susceptible  to  artifactual  disruption. 
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Asymmetry  of  Slow  Potential  Expectancy  Waves--In  some  situations, 
slow  potential  changes  in  the  brain  indicated  a change  in  functional  state 
more  clearly  than  spontaneous  oscillatory  EEG  activity.  The  contingent 
negative  variation  (CNV)  appears  to  reflect  increased  attentiveness  and 
arousal,  but  the  vertex  CNV  correlates  poorly  with  reaction  time.  We 
hypothesized  that  asymmetries  of  CNVs  might  be  related  to  hemispheric  cog- 
nitive differences  and  might  be  related  more  closely  to  discriminative  per- 
formance and  reaction  time  than  the  vertex  wave.  Cognitive  sets  were  in- 
cluded by  using  as  the  warning  stimulus  either  words  or  patterns  and  as 
imperative  stimuli  either  antonyms  or  synonyms  of  the  words  or  the  same 
or  different  patterns.  Subjects  were  required  to  make  differential  motor 
responses  to  imperative  stimuli  that  were  the  same  or  different  than  the 
warning  stimulus.  Parietal,  frontal,  and  vertex  placements  were  studied. 


In  parietal  leads,  CNVS  tended  to  be  larger  in  the  left  hemi- 
sphere preceding  words  and  larger  on  the  right  preceding  patterns,  but 
the  Task  X hemisphere  interaction  was  not  statistically  significant. 

That  interaction  was  much  clearer  for  fron to- temporal  leads  and  was  sig- 
nificant. Trends  related  to  slow  and  fast  reaction  time  trials  suggested 
that  performance  is  poor  when  interhemispheric  arousal  is  patterned  con- 
trary to  the  cognitive  demands  of  the  task. 


Analysis  of  Standard  Alpha  Band- -The  EEG  band  from  9 to  12  Hz 
obtained  in  the  original  EEG-per formance  study  was  analyzed  to  check  the 
assumption  that  establishing  alpha  bands  on  an  individual  basis  was  ad- 
vantageous. No  differences  in  EEG-performance  relationships  were  noted 
when  the  two  methods  of  establishing  alpha  were  used. 


Evaluation  of  one  Subject  with  Possible  Reversed  Dominance-- 


One  subject  s EPs  to  patterns  were  largest  in  the  left  hemisphere,  a 
configuration  shown  by  no  other  subject;  consequently,  her  data  were 
scrutinized  for  other  anomalies.  Of  15  measures,  she  showed  trends  op- 
posite the  mean  on  9 of  them,  whereas  the  highest  number  of  anomalies  for 
any  other  subject  was  6.  So  that  we  could  more  adequately  evaluate  her 
dominance  pattern,  she  was  tested  in  a dichotic  listening  task.  The  re- 
sults were  ambiguous--she  showed  a right- ear  (left  hemisphere)  superiority, 
but  it  was  a very  slight  one.  We  concluded  that  the  subject  was  not 
clearly  left  dominant  and  revised  her  performance  data  to  determine  the 
effect  on  mean  scores.  No  effect  was  noted  on  the  data  for  spatial  pat- 
terns--i.e. , the  EEG  performance  trends  were  unchanged.  However,  for 
verbal  data,  the  performance  curve  was  made  more  linear,  and  the  extreme 
scores  became  significantly  different.  These  results  suggest  that  the 
pattern  of  EP  distribution  between  the  hemispheres  might  be  a very 
sensitive  index  of  cerebral  dominance. 


GENERAL  INTRODUCTION 


This  project  was  initiated  to  test  the  feasibility  of  designing  a 
close- coup led,  two-way  communication  link  between  man  and  computer  using 
biological  information.  Specifically,  experiments  were  devised  to 
determine  whether  a computer  can  process  this  information  meaningfully 
and  whether  similar  biological  processes  representing  the  same  or  other 
thoughts  can  be  induced  in  the  same  or  another  individual. 

Possible  applications  of  such  a close-coupling  between  man  and 
machine  would  include  extremely  rapid  interactive  processing  between  a 
man  and  a computer  or  communication  between  two  or  more  persons  with  the 
computer  acting  as  an  interface.  For  example,  an  individual  using  such 
a biocybernetic  communication  system  would  be  able  to  "talk"  (i.e., 
both  send  and  receive)  with  a computer  at  the  speed  of  thought  rather 
than  be  limited  by  the  speed  of  a teletype  or  other  electromechanical 
device  through  which  ideas  in  the  form  of  questions  and  answers  normally 
must  pass.  In  addition,  nonverbal  imagery  and  affective  (emotional  or 
"feeling")  states  might  be  used  similarly  in  the  communication  process, 
thereby  significantly  increasing  the  bandwidth  of  information  transfer. 
Furthermore,  two  or  more  individuals,  separated  by  short  or  long  distances, 
would  have  the  capability  of  rapid  and  acc  rate  communication  with  a high 
degree  of  immunity  to  decoding  if  the  signals  were  intercepted,  where 
information  transfer  might  be  more  complete  than  with  normal  speech. 


Rationale  of  Aunroach,  Biocybernetic  Communication 

Our  approach  was  predicated  on  previous  research  conducted  by  the 
authors  and  others  in  the  areas  of  psychophysiological  measures  of 
thought,  computer  processing  of  electrophysiological  information,  and 
development  of  computer  pattern  recognition  techniques.  This  research 
may  be  summarized  as  follows. 

Early  work  by  Watson  (1930)  indicated  that  verbal  cognitive  processes 
may  be  represented  in  muscle  activity  of  the  vocal  apparatus  as  subvocal 
speech.  McGuigan  (1970),  reviewing  studies  of  such  covert  oral  behavior 
during  the  silent  performance  of  a language  task,  concludes  that  covert 
oral  behavior  (as  measured  by  the  electromyograph,  or  EMG)  increases 
significantly  in  amount  and  frequency  of  occurrence.  Thus,  verbal  ideas 
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or  thinking,  although  unquestionably  a central  nervous  system  process 
(MacNeilage  and  MacNeilage,  1971),  has  some  sort  of  peripheral  represen- 
tation in  the  muscles  of  the  vocal  apparatus. 

If  the  patterns  of  this  muscle  activity  are  at  all  similar  to  those 
involved  In  normal  overt  speech,  it  is  reasonable  to  assume  that  the 
electrical  activity  of  the  brain  during  covert  speech,  or  thinking,  may 
be  similar  to  tnat  during  overt  speech.  That  is,  a measure  of  the  scalp- 
recorded  electroencephalograph  (EEG)  of  a human  during  verbal  thinking 
could  be  similar  to  the  EEG  of  the  same  individual  when  expressing  the 
same  thoughts  vocally. 

However,  examination  of  the  "raw"  EEG  has  not  revealed  any  obvious 
pattern  related  to  overt  or  covert  speech;  it  may  be  that  only  patterns 
of  EEG  activity  between  various  areas  of  the  brain  at  a given  moment  are 
related  to  speech.  Several  technical  advances  made  in  recent  years  have 
provided  us  with  some  tools  to  deal  with  this  possibility.  Most  impor- 
tant is  the  use  of  computer  techniques  for  frequency  analysis  of  the  real- 
time EEG  and  the  development  of  multivariate  statistical  procedures 
(Donchin  and  Lindsley,  1966;  John  et  al.,  1964;  and  Rose  and  Lindsley, 
1965).  These  procedures  allow  comparison  of  specific  components  of  EEG 
waveforms  that  are  known  to  reflect  different  neurophysiological  pro- 
cesses. In  addition,  certain  statistics,  such  as  auto-  and  cross-spectral 
frequency  analysis  (Walter,  1963;  Walter  and  Adey,  1965),  linear  coherence 
function  (Adey,  Kado,  and  Walter,  1967),  and  the  weighted-average  coher- 
ence (Galbraith,  1967),  may  be  used  to  determine  t.he  degree  of  inter- 
action between  two  different  brain  regions.  Thus,  with  these  tools,  we 
can  examine  the  EEG  waveforms  from  several  areas  of  the  brain  that  are 
neurophysiologically  related  to  speech  to  determine  whether  their  pat- 
terns or  interaction  are  similar  during  overt  speech  and  verbal  thinking. 

A thorough  visual  analysis  of  the  statistical  results  of  these  EEG 
waveforms  would  be  extremely  complicated  and  time-consuming;  certainly 
on-line  visual  analysis  of  verbal  thinking  would  not  be  possible.  There- 
fore, we  turned  to  machine  pattern  recognition  techniques  to  analyze  the 
patterns  of  the  EEG  interrelationships  to  be  found  in  the  cross-spectra 
and  coherence  functions  related  to  covert  and  overt  speech.  Most  useful 
for  this  feasibility  study  were  techniques  for  cn-line  pattern  recogni- 
tion using  interactive  graphic  displays  (Hall  et  al.,  1968).  These 
techniques  allow  the  user  to  process  multivariate  data  by  using  all 
reasonably  conceivable  graphic  plots  and  further  to  manipulate  the  data 
using  appropriate  numeric  procedures  available  in  the  computer  system. 

Thus,  for  our  purposes,  a set  of  statistics  such  as  the  coherence  functions 
of  the  EEG,  the  patterns  of  the  EMG  changes  with  overt  speech,  and  other 
measures  may  be  plotted  as  a function  of  each  other  for  specific  covert 
language  tasks  (i.e.,  thinking). 
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Rationale  of  Approach,  Hemispheric  LafPral^y 

This  research  was  predicated  on  the  assumption  that  verbal  thinking 
is  nothing  more  than  covert  speech.  There  is  no  a priori  evidence  that 
he  cortical  organization  for  language  production  or  verbal  thinking  re- 
sides only  in  the  sites  we  have  chosen  to  place  our  EEC  recording  elec- 
trodes. Indeed,  evidence  developed  by  others  (described  below)  indicates 
at,  in  some  people,  cortical  organization  for  speech  may  occur  in  either 
left,  right,  or  both  hemispheres  and  in  other  cortical  locations  than 
those  nyestigated  here.  Our  results  wit’  dichotic  listening  tests  show 
a significant  positive  correlation  (0.786,  0.01  < p < 0.05)  between  com- 
puter correct  classification  of  EEC  responses  and  hemispheric  dominance 
tor  auditory  word  perception. 

For  those  reasons,  two  extensive  investigations  were  conducted  in 
addition  to  the  basic  study  of  EEC  pattern  recognition  during  overt  and 
coyert  speech,  to  determine  the  effects  of  laterality,  since  any  practi- 
cal biocybernetic  communication  system  must  consider  these  differences 
The  first  study  was  concerned  with  comparing  sujects  of  known  laterality 
ferences  for  speech,  on  speech  and  nonspeech  tasks,  to  determine  how 
patterns  of  EEG  related  to  speech  change  with  laterality.  The  second 
was  concerned  with  asymmetry  of  the  brain  in  language  and  nonlanguage 
tasks  and  its  relation  to  performance. 
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PART  I — B IOCYBERNET IC  COMMUNICATION 


Introduction 


The  biocybernetic  communication  parts  of  this  project  were  accom- 
plished in  two  stages:  (1)  off-line  computer  analysis  and  classification 
of  the  EEG  during  overt  and  covert  speech  and  (2)  on-line,  real-time 
classification. 


Off-Line  Analysis,  Overt  Classification 

Subjects 

Subjects  were  three  adult,  right-handed,  human  female  volunteers, 
ranging  in  age  from  21  to  41  year.:.  These  subjects  are  hereinafter  desig- 
nated B,  C,  and  D.  A total  of  six  experimental  sessions,  each  of  about 
2-1/2  hours  duration,  were  all  carried  out  using  the  same  experimental 
paradigm.  A given  session  for  a given  S is  identified  by  the  S's  letter 
code  and  her  chronological  session;  thus,  C5  was  the  fifth  experimental 
session  for  Subject  C.  Before  conducting  these  sessions,  several  appara- 
tus debugging  sessions  were  carried  out  with  a fourth  S,  A. 


Apparatus 

Electrodes  and  Electrode  Placements 


For  surface  recording  of  the  EMG  from  facial  muscles  involved 
in  speech  production,  Beckman  silver,  silver-chloride  miniature  disk  skin 
electrodes  (2-mm  exposed)  were  used.  EEG  scalp  electrodes,  reference 
electrodes,  and  the  ground  electrode  were  Beckman  silver,  silver-chloride 
standard  disk  skin  electrodes  (8-mm  exposed).  Two  reference  sites  were 
employed — the  skin  under  the  left  mastoid  for  EMG  recordings  and  the  skin 
under  the  right  mastoid  for  EEG  recordings.  All  recordings  were  monopolar 
to  record  absolute  potentials  at  the  recording  site. 

Selected  skin  areas  were  first  cleaned  with  acetone  (alcohol  on 
the  face),  then  conditioned  with  Redox  electrode  paste  by  rubbing  it  into 
the  skin,  and  followed  by  a second  cleaning  with  acetone.  A conductive, 
paste-filled  electrode  was  then  placed  over  each  recording  area  and 
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attached  by  a sticky  collar  to  the  underlying  skin.  Following  a record- 
ing session,  electrodes  were  removed,  and  the  skin  was  cleaned  with  ace- 
tone or  alcohol. 


■ 


Figure  1 shows  the  facial  musculature.  Muscles  involved  in 
vocalization  that  are  surface-recordable  are  2,  3,  4,  6,  7,  8,  9,  10,  11, 
13,  and  16.  Each  of  these  locations  was  tested  during  preliminary  experi- 
ments. In  Experiment  1,  combined  sites  13/16  and  7/8  were  found  to  produce 
the  most  reliable  integrated  EMG  patterns  during  overt  speech. 


Figure  2 illustrates  the  10/20  system  of  EEG  recording  (Penfield 
and  Jasper,  1954).  Locations  F7,  anterior  C5,  T5,  and  T6  were  used.  Three 
of  these  placements  overlie  areas  thought  to  be  involved  in  speech  (Pen- 
field  and  Roberts,  1959)  as  follows:  Broca's  speech  area;  anterior  C5, 

motor  control  of  vocal  musculature;  and  T5,  Wierneki's  area  for  speech 
organization  and  comprehension.  In  addition,  location  T6  on  the  right 
(nondominant)  hemisphere,  the  homolog  of  T5  over  the  dominant  hemisphere, 
was  employed  as  a control. 


After  electrodes  were  attached,  Ss  were  seated  in  a semidark, 
electrically  shielded  booth.  All  electrodes  were  plugged  into  a junction 
box  leading  to  the  recording  equipment.  Electrode  resistances  were  checked; 
if  any  one  electrode  was  found  to  be  greater  than  5,000  ohms,  it  was  re- 
moved, the  skin  was  cleaned  further  and  conditioned,  and  the  electrode  was 
replaced.  When  all  electrodes  checked  correctly,  the  was  instructed  in 
the  experimental  prodedure,  a microphone  for  recording  speech  was  placed 
in  front  of  the  £'s  mouth,  and  the  recording  session  was  begun. 


Equipment 


Figure  3 is  a diagram  of  the  equipment  setup.  Electrodes  from 
the  facial  musculature  were  led  first  to  a Beckman  Model  9852A  EMG  inte- 
grator coupler,  with  a fixed  time  constant  of  0.25  sec  and  pass  band  of 
20  to  5,000  Hz.  Channels  1 and  2 of  the  Dynograph  were  used  to  record 
the  integrated  EMG,  EEG  electrodes  were  led  to  Beckman-type  9806A  cou- 
piers,  with  the  pass  band  set  at  2 to  22  Hz  flat;  Channels  3,  4,  5,  and  6 
recorded  the  instantaneous  EEG.  Channel  7 recorded  the  voice  output  of 
the  microphone;  Channel  8 was  not  used.  Amplitude  normalization  was 
carried  out  by  setting  all  like  channels  (EMG  or  EEG)  to  the  same  gain. 

All  physiological  signals  were  preamplified  by  Beckman  Model  481B 
preamplifiers  and  then  were  led  in  parallel  to  Beckman  Model  482A  power 
amplifiers  with  calibrated  zero  suppression  and  to  an  Ampex  SP-300,  seven- 
channel,  analog  instrument  tape  recorder.  The  output  of  the  Beckman  power 
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FIGURE  1 MUSCLES  OF  THE  FACE  (AFTER  VAN  RIPER  AND  IRWIN,  1958) 


1.  Orbicularis  oculi  m.  (right) 

2.  Quadratus  labii  superioris  m. 

(right) 

3.  Zygomatic  head  of  quadratus 

labii  superioris  m.  (right) 

4.  Zygomaticus  m.  (right) 

5.  Risorius  m.  (right,  cut) 

6.  Triangularis  m.  (right) 

7.  Quadratus  labii  inferioris  m. 

(light) 


8.  Mentaiis  m. 

9.  Quadratus  labii  inferioris  m. 

(left) 

10.  Triangularis  m.  (left,  cut) 

11.  Zygomaticus  m.  (left,  cut) 

12.  Quadratus  labii  superioris 

m.  (left,  cut) 

13.  Orbicularis  oris  m. 

14.  Caninus  m.  (left) 


15  Buccinator  m.  (left) 

16.  Depressor  septi  nasi  m. 

17.  Nasalis  m.  (left) 

18.  Procerus  m. 

19.  Frontalis  m.  (left) 

20.  Frontalis  m.  (right) 

21.  Orbicularis  oculi  m. 

(left) 

22.  Nasalis  m.  (right) 
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amplifiers  drove  ink-writing  galvanometers  on  chart  paper  moving  at  25  mm/ 
sec.  EMG  and  EEG  recordings  were  on  Channels  1 through  6 of  the  Ampex, 
using  frequency  modulation,  at  1-7/8  in. /sec  (pass  band  dc-312  Hz);  voice 
was  recorded  on  Channel  7. 


Data  from  this  group  of  subjects  filled  one  10-1/2  in.  Ampex 
tape  with  analog  data.  These  data  were  then  sent  through  the  data  analy- 
sis system  using  the  Linc-8  laboratory  instrument  computer  and  a CDC-6400 
computer  (see  Data  Analysis  section  below). 


Procedure 


Language  Task 

Fifteen  words  were  selected  that  had  the  greatest  likelihood  of 
reproducing  reliable  EMG  patterns.  The  15  words,  shown  in  Table  1,  con- 
sisted of  five  monosyllabic  and  five  bisyllabic  words.  The  latter  pho- 
netically balanced  words  were  used  in  two  groups;  one  group  had  the  accent 
on  the  first  syllable,  and  the  other  groun  had  the  accent  on  the  second 
syllable.  These  15  words  were  chosen  (1)  to  emphasize  rounded  lips, 
spreading  lips,  bilabials,  and  open  lips  in  the  case  of  the  monosyllabic 
words  and  (2)  to  assess  the  effect  of  emphasis  (pre-  and  post-)  of  one 
syllable  on  the  other  in  the  case  of  the  bisyllabic  words.  No  covert 
responses  were  obtained  in  this  group  of  experiments. 


Table  1 

LANGUAGE  TASK  WORDS 


Bisyllabic 

Mono- 

Accent  Accent 

syl labic  First  Syllable  Second  Syllable 


TIP 

BLACKBOARD 

BLACKBOARD 

HIT 

SCHOOLBOY 

SCHOOLBOY 

HAD 

COUGHDROP 

COUGHDROP 

PUT 

SHIPWRECK 

SHIPWRECK 

COOL 

MOUSETRAP 

MOUSETRAP 
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Stimulus  Presentation 

Each  of  the  individual  words  in  Table  1 was  printed  on  a 35-mm 
slide  (white  on  black  to  reduce  glare)  and  presented  to  the  £ by  project- 
ing the  word  on  a rear-projection  screen  about  3 ft  from  the  £'s  eyes. 

The  subtended  visual  angle  of  the  stimulus  and  its  intensity  in  the  semi- 
darkened  room  were  chosen  to  avoid  squinting,  glare,  or  eye  strain  and 
to  reduce  eye  movements. 

After  installation  in  the  recording  chamber,  the  £ was  instructed 
that  she  was  to  relax  with  eyes  closed  while  the  polygraph  and  tape  re- 
corder gains  and  filters  were  adjusted  for  proper  EMG  and  EEG  recordings. 
During  that  period,  the  £ was  to  say  her  name  when  asked  (to  calibrate 
EMG  gains  ard  the  voice  channel)  and  to  open  or  close  her  eyes  when  asked 
(to  check  for  alpha  in  the  closed-eyes  EEG  and  alpha  blocking,  or  desyn- 
chronization, with  the  eyes  open  and  to  check  for  eye  movement  artifact). 
Following  these  adjustments,  the  £ was  cold  that  she  would  be  presented 
with  a list  of  15  words,  one  at  a time,  for  ten  full  presentations.  The 
presentations  would  be  visual.  (The  £ was  shown  a test  word  on  the  screen 
as  an  example.)  The  £ was  to  sit  relaxed  with  her  eyes  closed.  On  hear- 
ing the  statement  "ready"  from  the  experimenter,  she  was  to  open  her  eyes 
and  look  at  the  screen.  In  2 to  3 sec,  a stimulus  word  would  be  projected 
on  the  screen  for  about  3 sec,  during  which  time  she  was  to  read  the  word 
aloud  into  the  microphone.  When  the  projected  word  was  turned  off,  she 
was  to  close  her  eyes  until  the  next  word  was  presented,  and  wait  until 
the  next  "ready"  signal.  The  15  words  were  presented  a total  of  ten  times 
per  recording  session  per  subject. 

Each  of  the  three  £s  was  run  a second  time  not  less  than  one 
week  nor  more  than  one  month  following  the  first  session.  This  set  of 
measurements  was  recorded  exactly  the  same  as  the  first  and  was  run  to 
determine  within  £ reliability.  Thus,  the  data  were  based  on  six  elec- 
trodes per  £,  for  three  £s,  two  sessions  each,  where  each  session  con- 
sisted of  ten  repetitions  of  15  words  and  one  sentence,  for  a total  of 
5,400  electrophysiological  responses. 


Data  Analysis 

Editing  and  Digitizing 

A synchronization  signal  on  Channel  7 (voice  channel)  of  the 
Ampex  analog  recorder,  which  preceded  each  stimulus  presentation,  caused 
the  Linc-8  to  begin  sampling  the  six  channels  of  data  and  the  voice  han- 
nel  through  analog-to-d igital  converters.  A total  of  7 sec  of  data  were 
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sampled  at  42  samples/sec  for  each  of  the  seven  channels.  The  seventh 
second  of  data  was  collected  for  time  justification  of  the  electrophysio- 
logical  response  as  described  below.  The  data  were  stored  in  memory,  and 
on  command  any  two  of  the  six  data  channels  or  the  voice  channel  could 
be  displayed  on  a two-channel,  cathode-ray  oscilloscope  driven  by  the  com- 
puter. The  display  itself  consisted  of  6 sec  of  data,  or  256  data  points. 
In  addition,  the  scope  displayed  a vertical  cursor  that  was  exactly  cen- 
tered to  represent  a zero  point  for  time  justification  of  the  electro- 
physiological  response. 

Figure  4 illustrates  the  seven-channel  dynograph  recording  of 
the  EMG  and  EEG  for  one  presentation  of  the  word  "COOL."  Note  the  cursor 
line  at  the  onset  of  vocalization  (Channel  7),  which  was  used  to  time- 
justify  all  electrode  responses.  Any  two  of  these  channels  could  be  dis- 
played on  the  Line-scope,  as  shown  in  Figure  5.  In  Figure  5A,  an  EMG 
response  is  shown  on  the  top  channel;  the  voice  voltage  and  the  centered 
cursor  are  shown  on  the  second  channel.  Note  that  the  vocal  onset  is 
off-centered  on  the  scope,  indicating  that  this  particular  response  was 
not  time  justified  for  vocalization  to  occur  at  exactly  3 sec  from  the 
onset  of  the  display.  By  use  of  a second  Linc-8  command,  all  six  channels 
of  data  and  the  voice  channel  could  be  rotated  simultaneously  into  and  out 
of  memory  with  the  extra  seventh  second  to  place  the  onset  of  vocalization 
at  any  desired  point.  This  feature  was  used  to  shift  the  data  and  voice 
channels  (to  the  left  in  this  case)  so  that  the  onset  of  vocalization  oc- 
curred at  the  centered,  3-sec  cursor,  as  shown  in  Figure  5B.  Thus,  after 
time  justification,  3 sec  of  data  were  displayed  before  the  onset  of 
vocalization  and  3 sec  after.  On  completion  of  time  justification,  the 
six  channels  of  data  were  stored  on  Line  tape  on  the  computer. 

When  responses  to  all  150  word  utterances  and  the  10  sentences 
for  a given  subject— session  had  been  stored  on  Line  tape  (six  channels 
per  word  and  sentence,  for  a total  of  990  tape  blocks),  another  Line— 8 
program  was  used  to  transfer  this  data  to  the  Pertec  digital  tape  recorder 
in  ten  files  of  101  records  per  file  (255  samples  per  block  were  trans- 
ferred rather  than  the  256  collected  because  of  an  error  in  the  transfer 
program).  The  Pertec  tape  was  then  unpacked  on  the  CDC-6400  computer, 
and  the  data  words  were  reordered  in  sequence  for  data  processing. 

CDC-6400  Response  Classification  by  Averaging 

As  a first  approximation  for  machine  pattern  recognition  of  the 
six  el ec trophys iolo gical  responses  for  each  word  utterance,  we  decided  to 
use  a method  of  averages  to  construct  templates  for  response-word  classi- 
fication. The  data  unit  of  ana  Lysis  for  classification  was  the  individual 


24 


CHANNEL 


SUBJECT  C6 


ELECTRODE 


1 EMG  13/16 


2 EMG  7/8 


4 EEG  T5 


5 EEG  C5 


6 EEG  T6 


1 VOICE  AND  MARKER 


BEGINNING 
OF  VOCALIZATION 


TIME  MARKS  (1  sec) 


'READY" 


CLOSED 


STIMULUS  VOCAL  RESPONSE 
ON  "COOL" 


FIGURE  4 DYNOGRAPH  RECORDING  OF  STIMULUS-RESPONSE  OF  EMG  AND  EEG 
FOR  THE  WORD  "COOL” 
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6-sec  electrode  response  corresponding  to  a word.  To  determine  whether 
the  C DC-64 00  computer  could  classify  correctly  the  word  by  analysis  of 
the  electrophys io logical  response  alone,  the  response  was  compared  with 
15  templates  (one  for  each  of  the  15  words). 

As  shown  in  Figure  6,  a given  template  was  the  average  electro- 
physiological,  6-sec  response  for  the  ten  repetitions  of  the  word  for 
a given  5 on  each  session.  Each  sample  point  of  the  255  samples  in  a 
6-sec  epoch  was  added  to  each  corresponding  point  of  the  other  nine  re- 
sponses, and  the  result  was  divided  by  ten  for  the  average  response  for 
that  point.  Since  there  were  six  electrodes  and  15  words,  there  were 
90  templates  altogether  for  each  on  each  session,  or  a total  of  540 
templates  for  the  three  Ss,  two  sessions  each. 

To  compare  a single  electrode  response  for  a single  word  ut- 
terance with  each  of  the  15  templates  for  that  electrode,  a root-mean- 
square  (RMS)  difference  was  calculated  between  the  single  response  and 
the  template,  as  shown  in  Figure  7.  In  the  top  of  the  figure,  the  EMG 
13/16  ulectrode  response  to  TIP  2 (second  utterance  of  TIP)  is  compared 
with  the  template  for  COOL.  That  is,  each  sample  of  the  255  samples  in 
the  response  was  subtracted  from  the  corresponding  sample  in  the  template. 
The  difference  for  each  sample  point  (only  a few  fictitious  values  were 
used  in  Figure  7 for  illustration)  was  squared,  and  the  squared  differ- 
ences were  summed  over  the  255  samples.  Dividing  by  255  and  taking  the 
square  root  provided  the  single  RMS  number  for  that  comparison. 

In  the  bottom  of  Figure  7,  the  same  electrode  response  to  TIP  2 
is  compared  with  the  template  for  the  word  TIP.  Since  TIP  2 was  included 
in  making  the  template  for  TIP  (see  Figure  6),  this  template  was  neces- 
sarily biased  in  favor  of  classifying  TIP  2 response  as  TIP.  Therefore, 
for  this  response  and  all  other  responses  compared  with  their  own  templates, 
the  individual  response  was  first  subtracted  from  the  template.  The  RMS 
difference  then  was  calculated  for  the  individual  response  against  the 
unbiased  average  response  for  the  other  nine  utterances  of  the  word. 

After  calculating  the  RMS  difference  for  a given  electrode  re- 
sponse for  a given  word  against  all  15  templates,  the  computer  then  clas- 
sified the  response  as  the  word  having  the  smallest  RMS  value.  For  ex- 
ample, in  Figure  7,  TIP  2 is  classified  as  "TIP"  rather  than  "COOL,"  since 
the  RMS  difference  with  TIP  AVERAGE  minus  TIP  2 is  less  than  the  RMS  dif- 
ference with  COOL  AVERAGE.  If  this  RMS  difference  remained  the  smallest 
when  TIP  2 was  compared  with  all  15  templates,  ultimately  the  TIP  2 re- 
sponse was  classified  as  TIP. 
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FIGURE  7 CLASSIFICATION  OF  TIP  2 AS  TIP  WITH  RESPECT  TO 
(COOL)  USING  RMS  DIFFERENCE 


ANOTHER  TEMPLATE 
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In  this  way,  response  classifications  were  made  for  each  of  the 
150  words  per  per  session  per  electrode  (Within  Subjects,  Within  Ses- 
sions classification).  Two  additional  classifications  also  were  made: 

(1)  Within  Subjects  and  Between  Sessions  (to  assess  within  S^  reliability); 
and  (2)  Between  Subjects  (to  assess  individual  differences).  In  the  first 
case,  the  six  electrode  responses  for  each  150  words  of  one  session  for 
a given  £ were  compared  with  the  15  templates  for  the  other  session  for 
that  £ (e.g.,  C5  responses  with  C6  templates,  and  C6  responses  with  C5 
templates).  In  the  second  case,  the  six  electrode  responses  for  each  of 
the  150  words  of  a given  S on  a given  session  were  compared  with  the  15 
templates  for  another  £ on  a given  session  (e.g.,  C5  responses  with  B5 
templates  or  D4  responses  with  C6  templates). 


Finally,  in  addition  to  obtaining  classification  of  responses 
for  each  electrode,  we  decided  to  pool  both  EMG  electrodes,  all  four  EEG 
electrodes,  and  all  six  electrodes  (EMG  plus  EEG)  to  assess  the  relative 
contribution  of  types  and  number  of  electrode  responses  to  the  computer 
classification. 


Determination  of  Critical  Classification  Period 

The  electrophysiological  recordings  shown  in  Figure  4 to  the 
word  COOL  illustrate  that  only  about  1 sec  of  the  6-sec  epoch  actually 
is  involved  in  making  the  response.  Since  computer  classification  of  an 
electrophysiological  response  possibly  might  be  based  on  that  portion  of 
the  response  following  stimulus  onset,  but  before  the  onset  of  vocaliza- 
tion, we  decided  to  determine  which  parts  of  the  6-sec  epoch  contributed 
to  the  classification  and  to  what  extent.  To  do  this,  an  F-ratio  was 
calculated  for  each  electrode  for  the  Within  Words  variance  and  Between 
Words  variance  for  each  of  the  255  data  samples  in  the  6-sec  epoch.  If 
a given  sample  point  was  not  contributing  to  the  computer  classification 
of  the  word,  the  ratio  of  the  two  variances  (i.e.,  the  F-statistic)  should 
be  small.  On  the  other  hand,  if  the  Between  Word  variance  was  signifi- 
cantly higher  than  the  Within  Word  variance  for  a given  sample  point,  we 
can  assume  with  some  confidence  (given  by  statistical  tables  for  the  F- 
ratio)  that  the  particular  data  point  was  contributing  to  the  classifi- 
cation. 


Examples  of  the  F-ratio  for  the  255  samples  in  the  6-sec  epoch 
for  an  EMG  and  an  EEG  electrode  are  shown  in  Figures  8 and  9,  respectively, 
for  £ C,  Session  5.  Note  that,  in  both  cases,  the  F-ratio  remains  small 
and  statistically  insignificant  for  about  the  first  100  samples  but  becomes 
and  remains  significantly  large  from  about  Sample  101  to  about  Sample  200. 
For  the  EEG  (Figure  9),  another  portion  between  about  Samples  205  through 
230  also  reaches  significance.  This  means  that,  for  both  EMG  and  EEG 
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responses  taken  separately,  only  that  portion  of  the  6-sec  epoch  between 
Samples  101  and  200  probably  contributes  significantly  to  classification 
of  the  response  (that  portion  of  the  EEC  from  205  to  230  also  may  contrib- 
ute but  less  significantly).  F-ratios  were  calculated  for  all  Ss,  sessions 
and  electrodes  with  essentially  the  same  results. 


CPC -6400  Response  Classification  by  Other  Statistics 

As  noted  earlier,  there  is  no  a priori  reason  for  believing  that 
a particular  statistic  of  the  EEC  and  EMG  ultimately  will  be  a better  clas- 
sifier of  the  verbal  response  than  any  other.  Consequently,  several  other 
statistics  were  computed,  RMS  differences  were  obtained  in  the  same  manner 
as  with  the  averaged  responses,  and  individual  electrode  responses  were 
c assif led.  These  statistics  were  auto-  and  cross-spectra,  linear  coher- 
ence, and  weighted-average  coherence.  These  statistics  are  presumed  to  be 
useful,  since  they  involve  frequency  analysis  of  the  EEG  and  may  be  used 
to  determine  the  degree  of  interaction  between  two  or  more  different  re- 
gions of  the  brain.  Such  an  analysis  should  provide  information  about 
cortical  organization  and,  therefore,  may  show  patterns  of  organization 
specific  for  particular  responses  to  the  15  stimulus  words.  Except  for 
the  actual  calculation  of  the  statistic  itself,  all  other  computations 

for  response  classification  were  the  same  as  for  the  averaged  responses 
described  above. 


Tests  for  Statistical  Significance 

To  show  that  correct  classification  of  single  electrode  responses 
by  the  computer  did  not  occur  by  chance,  a chi  square  was  calculated  for 
each  electrode  subclassification  for  each  S and  each  session,  between 
sessions  and  between  subjects.  The  "expected"  frequencies  for  evaluating 

the  chi  square  were  based  on  classifications  being  randomly  distributed 
across  words. 


Control  of  Artifact 


In  a working  biocybernetic  communication  system,  constraints 
on  t e user  against  eye  or  muscle  movement  may  not  be  effective.  Since 
such  movements  often  produce  artifacts  in  electrophys iological  responses 
(especially  in  the  EEG),  it  is  imperative  that  a system  be  designed  that 
is  not  unduly  affected  by  such  artifacts.  That  is,  pattern  recognition 
and  classification  should  be  carried  out  successfully  whether  or  not 
artifacts  are  present.  For  this  reason,  we  decided  to  analyze  these  data 
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without  removing  those  responses  with  known  artifacts  present;  that  is, 
this  was  a "worst-case"  analysis. 

Nevertheless,  some  effort  was  devoted  to  controlling  the  identi- 
fying artifacts  and  to  assessing  their  contribution  to  response  classifi- 
cation. This  was  accomplished  by  obtaining  responses  in  the  four  EEG 
electrodes  to  five  words  under  conditions  of  both  visual  and  auditory 
stimuli  and  with  and  without  known  eye  movements  and  movement  artifacts. 
Finally,  during  data  collection,  Ss  were  instructed  to  remain  as  relaxed 
as  possible  during  the  response  period,  with  no  more  eye  or  body  movement 
than  necessary.  An  analysis  of  all  artifactual  data  showed  that  none  of 
the  various  types  of  artifact  contributed  significantly  to  correct  com- 
puter classifications,  although  they  did  contribute  to  incorrect  classifi- 
cation (i.e.,  confusion). 


Results 


Classification  by  Averages 

Individual  Responses — Figure  10  illustrates  the  six  electrode 
templates  for  5>  C5  for  the  word  C0UGHDR0P  with  the  accent  on  the  first 
syllable.  Note  that,  for  both  EMG  and  EEG  records,  significant  changes 
occur  primarily  over  computer  Samples  101  to  200,  thus  beginning  about 
0.5  sec  before  onset  of  vocalization  (arrow  at  3.0  sec). 

Figure  11  shows  the  variability  around  the  averages  of  Figure  10, 
the  upper  trace  of  each  pair  being  the  average  plus  one  standard  devia- 
tion, the  lower  trace  the  average  minus  one  standard  deviation.  Note  that 
the  variability  of  each  electrode  signal  is  relatively  small.  Similar 
plots  were  made  for  all  three  Ss  for  the  two  sessions,  each  with  essen- 
tially similar  results,  indicating  that  it  should  be  feasible  to  identify 
a given  spoken  word  by  comparing  the  electrophysiological  response  to  the 
templates. 


As  described  in  the  previous  section,  for  each  on  each  session, 
the  individual  electrode  response  for  each  of  the  150  word  utterances  was 
compared  with  the  15  templates.  These  comparisons  resulted  in  15  RMS 
values,  from  which  the  CDC-6400  computer  classified  the  response  with  the 
word  for  which  the  RMS  value  was  a minimum.  Tables  of  correct  responses 
were  then  constructed  for  each  j3  and  each  session,  showing  the  number  of 
correct  classifications  for  the  150  words  for  each  electrode  and  each 
subc lass if ication. 
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FIGURE  10  TEMPLATES  (AVERAGES  OF  10  RESPONSES  EACH)  FOR  THE  SIX 
ELECTRODES,  SUBJECT  C5,  FOR  THE  WORD  COUGHDROP 
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FIGURE  11  VARIABILITY  OF  AVERAGE  RESPONSES  OF  THE  SIX  ELECTRODES, 
SUBJECT  C5,  FOR  THE  WORD  COUGHDROP 


Within  Subjects  Resuits--To  test  the  reliability  of  measurements 
and  response  classification,  all  three  Ss  were  run  on  two  sessions  each. 

The  sessions  were  no  less  than  one  week  and  no  longer  than  one  month  apart. 
Tables  of  correct  classifications  were  constructed  for  each  S and  each  ses- 
sion and  comparisons  were  made  for  both  Within  Subjects/Within  Sessions, 
and  Within  Subjects/Between  Sessions. 

Table  2 summarizes  the  Within  and  Between  Sessions  comparisons 
for  S^  C,  for  the  best  classifier  subgroup  (101-200)  for  all  electrode 
combinations  (similar  tables  were  constructed  for  Ss  B and  D) . The  nearest 
percentage  of  correct  responses  and  the  rank  order  of  electrodes  as  classi- 
fiers are  given  by  classifying  each  response  first  with  the  templates  of 
its  own  session  and  then  with  the  templates  from  the  other  session.  The 
following  may  be  concluded  from  inspection  of  Table  2,  and  similar  results 
from  Ss  B and  D: 

(1)  Chance  expectancy  was  10%,  and  significant  classifica- 
tion rates  (p  = 0.05'/  were  at  25%.  Thus,  all  correct 
classification  rates  within  sessions  were  statistically 
significant. 

(2)  In  all  three  Ss , a greater  number  of  responses  were 
classified  correctly  on  the  second  session  (Within 
Sessions  comparison)  than  on  the  first  (by  rank  order 
and  not  necessarily  for  a particular  electrode  or 
combination) . Thus,  an  habituation  or  learning  effect 
is  present  that,  presumably,  reduces  the  variability 
from  one  session  to  the  next. 

(3)  In  all  three  _Ss,  when  the  second  session  responses  are 
compared  with  the  first  session  templates  (Between 
Sessions  comparisons),  a greater  number  of  responses 
were  classified  correctly  than  when  the  first  session 
responses  are  compared  with  the  second  session  tem- 
plates. This  is  also  probably  due  to  a decreased 
variability  in  the  individual  responses  of  the  second 
session,  further  strengthening  the  conclusion  that 
habituation  or  learning  operates  to  improve  performance 
in  response  classification. 

(4)  In  all  three  S_s , the  EWG  responses,  taken  separately 
or  together,  are  better  classifiers  than  any  EEG  re- 
sponse or  the  EEG  responses  taken  together.  However, 
there  appears  to  be  no  consistency,  either  within  a 

S or  across  Ss,  for  any  particular  EEG  response  being 
a better  classifier  than  any  other. 


Table  2 


NEAREST  PERCENTAGE  OF  CORRECT  RESPONSES  AND  RANK  ORDER 
OF  ELECTRODES  AS  CLASSIFIERS,  SAMPLE  POINTS  101-200, 
SUBJECT  C,  WITHIN  AND  BETWEEN  SESSIONS 


Within  Sessions  Between  Sessions 


C5  with  C5* 

C6  with 

C6 

C5  with 

C6t 

C6  with 

C5 

Channel 

Percent 

Correct 

Rank 

Percent 

Correct 

Rank 

Percent 

Correct 

Rank 

Percent 

Correct 

Rank 

13/16 

1 

57% 

3 

76% 

2 

39% 

3 

53% 

3 

7/8 

2 

43 

7 

56 

4 

33 

4 

40 

4 

F7 

3 

45 

6 

48 

5 

22 

7 

15 

9 

T5 

4 

47 

5 

46 

7 

31 

6 

29 

6 

C5 

5 

39 

8 

36 

9 

14 

9 

28 

7 

T6 

6 

26 

9 

37 

8 

21 

8 

25 

8 

EMG 

63 

2 

79 

1 

42 

2 

64 

2 

EEG 

49 

4 

47 

6 

31 

5 

31 

5 

All 

70 

1 

75 

3 

50 

1 

65 

1 

C5  data  compared  with  C5  templates. 
C5  data  compared  with  C6  templates. 


(5)  The  range  of  percentages  of  correct  classifications 
across  £s  is  from  9 to  84 7«,  with  both  EMG  and  all 
six  electrodes  generally  in  the  higher  ranges,  EMGs 
separately  and  all  EEGs  in  the  middle  range,  and 
EEGs  separately  in  the  lower  ranges. 

(6)  Comparisons  among  percentages  of  correct  responses 
among  the  three  ^5s  indicate  that  responses  from  J>  C 
are  better  classifiers,  in  general,  than  either  B's 
or  D's,  with  B slightly  better  than  D.  Examination 
of  either  raw  data  or  templates  shows  that  this  re- 
sult is  probably  due  to  variability  (larger  in  D, 
less  in  B,  still  less  in  C) . 


Between  Subjects  Results — The  results  of  the  EMG  analysis  showed 
that  an  individual's  electrophys iologlcal  responses  accompanying  speech 
are  unique  to  that  individual.  Therefore,  a comparison  was  made  Between 
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Subjects  for  one  session  for  all  combinations.  The  results  of  this  com- 
parison are  shown  in  Table  3,  which  shows  that  even  though  the  EMG  re- 
sponses are  significant  at  p < 0.01,  the  highest  frequency  (both  EMG)  is 
only  39%,  compared  with  747.  Within  Subjects/Within  Sessions,  and  62% 
Within  Subjects/Between  Sessions.  These  results  suggest  that  homologous 
muscles  act  more  like  each  other  between  Ss  than  homologous  brain  sites. 
Table  3 also  shows  that,  although  EMGs  taken  separately  or  together  are 
better  classifiers  than  the  EEGs  taken  separately  or  together,  no  EEG 
electrode  is  preferred  for  classification  purposes. 


Table  3 


NEAREST  PERCENTAGE  OF  CORRECT  RESPONSES  AND  RANK  ORDER 
OF  ELECTRODES  AS  CLASSIFIERS,  SAMPLE  POINTS  101-200, 
ALL  SUBJECTS,  ALL  SESSIONS 


Within 
Sessions/ 
Within  Subiects 

Between 
Sess ions/ 
Within  Subiects 

Between 

Subject! 

s 

Channel 

Percent 

Correct  Rank 

Percent 

Correct 

Rank 

Percent 

Correct 

Rank 

1 

63% 

3 

48% 

4 

27% 

2 

2 

59 

4 

51 

3 

24 

3 

3 

28 

8 

17 

9 

8 

6.5 

4 

35 

5 

22 

7.5 

7 

8.5 

5 

30 

7 

22 

7.5 

7 

8.5 

6 

28 

9 

24 

5 

8 

6.5 

EMG 

74 

1 

62 

1 

39 

1 

EEG 

33 

6 

24 

6 

9 

5 

Ml 

65 

2 

58 

2 

22 

4 

Sources  of  Error 

As  indicated  under  "Data  Analysis,"  we  made  no  special  attempts 
to  remove  artifacts  or  other  known  sources  of  error  because  we  wanted  to 
see  how  well  the  computer  would  perform  in  classifying  responses  under  the 
worst-case"  condition.  Several  known  sources  of  error  existed,  including 
time  and  amplitude  variations  and  muscle  and  eye  movement  artifacts.  Each 
of  these  is  discussed  below,  and  their  relative  contribution  is  evaluated. 


39 


Confusion  Matrices  (Bisyllabic  Confusion) — To  determine  why 
those  responses  not  correctly  classified  were  confused  with  other  tem- 
plates, a set  of  confusion  matrices  was  constructed  for  each  electrode, 
for  each  S,  and  for  each  session  showing  how  each  electrode  response  was 
classified.  This  is  shown  for  all  Ss  in  Table  4.  Note  that,  for  computer 
Samples  1 through  100  and  200  through  255  (left  side  of  table),  the  rates 
of  correct  classifications  are  quite  low  and  appear  to  be  randomly  dis- 
tributed. This  is  in  contrast  with  Subgroups  101  through  200  and  1 through 
255  (right  side  of  table),  where  a strong  diagonal  begins  at  the  upper 
left,  each  cell  of  the  diagonal  representing  the  number  of  correct  re- 
sponses . 


There  are  also  two  strong  subdiagonals  in  Subgroups  101  through 
200  and  1 through  255.  The  first  begins  at  the  junction  of  SB2  (column) 
and  SB1  (row);  the  second  begins  at  the  junction  of  SE1  (column)  and  SB2 
(row).  These  subdiagonals  represent  a strong  confusion  by  the  computer 
between  two  identical  bisyllabic  words  with  different  accents,  suggesting 
that  the  effect  of  emphasis  on  classification  is  not  large.  This  result 
further  suggests  that  the  percentage  of  correct  classification  can  be 
increased  materially  if  emphasis  is  not  taken  into  account  by  the  computer. 
That  is,  we  can  regard  confusions  of  emphasis  not  as  errors,  but  as  cor- 
rect classifications  of  responses.  For  example,  the  number  of  total  cor- 
rect responses  in  the  main  diagonals  for  Subgroups  101  to  200  in  Table  4 
is  592,  or  63%.  By  disregarding  the  confusions  of  emphasis  on  the  bisyl- 
labic words  in  classifying  responses,  this  figure  increases  to  689,  or  76%, 
correctly  classified  responses. 

Time  Shift  and  Amplitude  Variation — Two  additional  sources  of 
error  are  illustrated  in  Figure  12.  The  first,  an  EMG  response  to  TIP  8, 
was  a slight  shifting  ii  time  (to  the  left)  of  the  electrophys iological 
response  with  respect  to  the  onset  of  vocalization.  Since  the  onset  of 
vocalization  was  taken  as  the  time  zero  reference,  then  any  such  shift 
would  cause  an  incorrect  classification,  even  though  the  waveform  of  the 
individual  response  is  very  similar  to  the  template.  In  this  case,  TIP  8 
was  confused  with  COOL.  The  second  source  of  error  was  due  to  amplitude 
variation,  as  shown  in  an  EMG  response  to  TIP  9,  so  that  TIP  9 also  was 
confused  with  COOL  (note  that  although  there  was  also  a time  shift  in 
TIP  9,  it  was  shifted  relative  to  both  templates).  However,  attempts  at 
machine  correction  for  the  first  type  of  error  were  relatively  unsuccess- 
ful in  improving  classification  rates,  largely  because  the  entire  signal 
had  to  be  time  justified,  thus  producing  errors  of  temporal  adjustment  in 
one  part  of  the  signal  while  correcting  errors  in  another.  Some  form  of 
dynamic  programming,  or  perhaps  the  use  of  multivariate  statistics,  may 
overcome  this  error. 
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FIGURE  12  TWO  EMG  RESPONSES  (ELECTRODE  13/16,  SUBJECT  C5)  IO  TIP, 
CONFUSED  WITH  COOL  DUE  TO  TIME  SHIFT  AND  AMPLITUDE 
VARIATION 


nother  source  of  temporal  error  was  contraction  or  expansion 
in  time  of  an  individual  response  with  respect  to  the  template.  A com- 
puter  program  designed  to  detect  such  errors  and  to  correct  for  them  was 
devised  but  did  not  significantly  change  correct  classification  rates. 


Classification  by  Other  Statistics 

In  addition  to  the  average  template  comparisons  described  above 
several  other  statistics  also  were  calculated  for  S C,  Session  5 includ-’ 
mg  linear  coherence  (Coh),  weighted-average  coherence  (Ave  Coh)  ’ and 
spectral  densities  (Freq)  for  auto-  and  cross-spectra.  The  results  were 
uniformly  disappointing  as  follows: 


Channels 


Both  EMG 

All  EEG 

All  electrodes 


Correct  Responses 


per  150 
Statistic 

Coh 

Ave  Coh 

Freq 

31 

14 

34 

17 

10 

8 

25 

17 

16 

Because  of  these  results  on  S C, 
the  other  two  Ss. 


these  statistics  were  not  calculated  for 


Since  all  three  statistics  are  based  on  frequency  analysis  we 
suspect  that  these  low  correct  classification  scores  are  due,  partly’at 

least,  to  differences  in  phase  that  were  not  taken  into  account  in  calcu- 
lating these  statistics. 


One  finding  in  the  spectral  density  plots  was  that  more  alpha 
activity  occurred  in  the  nondominant  right  hemisphere  (EEG  T6)  than  in 
the  homologous  electrode  (EEG  T5)  over  the  dominant  (speech)  hemisphere. 
Since,  in  this  S,  T6  (nondominant)  responses  were  the  poorest  EEG  classi- 
fiers and  T5  responses  were  the  best  EEG  classifiers,  this  alpha  different 

may  be  related  to  the  variability  in  the  two  templates  for  response  class! 
fication. 
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Analysis  of  Covert  Speech 


Introduction 


A primary  objective  of  this  research  project  was  to  demonstrate  the 
feasibility  of  man-machine  communication  through  verbal  thinking  and  thus 
bypass  the  normal  input-output  mechanisms.  A major  operating  assumption 
has  been  that  verbal  thinking  is  nothing  more  than  subvocal  speech,  and 
that  therefore  the  electrical  activity  of  the  brain  during  covert  speech 
may  be  similar  to  that  during  overt  speech.  Our  approach  the  first  year 
was  to  demonstrate  that  unique  patterns  of  electrical  activity  of  the 
brain  exist  during  overt  speech,  and  that  a computer  pattern  recognition 
technique  could  be  used  to  identify  such  activity  when  it  occurs. 

It  remained  to  be  demonstrated  that  the  electrical  activity  of  the 
brain  during  covert  speech  resembled  that  of  overt  speech  when  the  same 
words  were  spoken  and  that  the  computer  pattern  recognition  technique 
could  equally  well  classify  EEG  responses  associated  with  words  covertly 
spoken.  The  results  of  experiments  demonstrating  this  effect  are  pre- 
sented here. 

Covert  speech  experiments  were  conducted  with  two  right-handed  female 
volunteers.  In  both  instances,  covert  speech  was  defined  as  silent  read- 
ing of  words  visually  projected  on  the  rear-vision  screen  as  described 
under  "Methods"  below.  We  recognized  that  "silent  reading"  may  not  be 
the  same  thing  as  "verbal  thinking,"  but  decided  that  it  would  be  best 
to  proceed  first  in  a nonvocal  experiment  that  was  as  similar  to  the 
vocal  experiments  as  possible. 


Methods 


The  procedure  was  as  outlined  in  "Procedure"  ("Off-Line  Analysis, 

Overt  Classification")  for  both  subjects,  except  that  the  stimulus  words 
were  read  silently.  For  S.  B7,  five  words  were  used  both  overtly  and 
covertly  for  comparison  purposes,  and  each  word  was  read  five  times  each 
in  both  conditions.  The  words  were:  PUT,  SCHOOLBOY,  COUGHDROP,  TIP, 

and  HAD.  For  S W3 , one  set  of  three  consonant-vowel-consonant  (CVC)  words, 
varying  only  in  the  first  consonant,  and  one  set  of  three  homonyms  (to 
compare  with  experiments  on  overt  production  of  homonyms  described  in  the 
next  section  below)  were  used.  Each  word  was  repeated  ten  times  each. 

The  words  were:  BAN,  PAN,  MAN,  WRITE,  RIGHT,  AND  RITE.  For  both  sub- 

jects, no  recordable  EMG  signals  could  be  obtained  from  the  facial  muscle 
electrodes  and  therefore  were  not  analyzed.  Horizontal  and  vertical  eye 
movements  were  recorded  for  S.  B7  to  identify  eye-movement  artifact  in  the 
EEG.  For  both  Ss,  the  four  EEG  electrodes  described  in  "Procedure"  (F7, 


J 
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T5,  C5,  and  T6)  were  employed.  For  _S  B7,  the  Linc-8  computer  was  used 
for  response  classification,  whereas  for  S W3,  the  CDC-6400  was  used 
initially  and  the  Linc-8  later.  In  both  cases,  a correlational  time 
justification  technique  was  employed,  since  no  onset  of  vocalization 
could  serve  as  a zero  reference. 


Results 

The  initial  results  were  mixed.  For  S_  B7,  the  percentage  of  correct 
classifications  for  the  covertly  spoken  words  was  about  the  same  as  that 
for  overtly  spoken  words  (277>  and  297>,  respectively,  for  all  four  EEG 
electrodes  taken  together).  Furthermore,  the  pattern  of  the  covert  re- 
sponses strongly  resembled  the  overt  responses  for  the  same  word  and 
electrode,  confirming  the  assumption  that  electrical  activity  of  the 
brain  during  covert  speech  (silent  reading)  is  similar  to  that  of  overt 
speech  of  the  same  words.  This  is  shown  clearly  in  Figure  13,  where 
single  responses  by  S B7  to  the  same  word  in  both  the  overt  and  covert 
conditions  are  illustrated  (nontime  justified).  Since  these  are  not 
averages,  the  faster  activity  of  the  normal  EEG  is  the  predominant  poten- 
tial waveform  from  all  electrodes.  Nevertheless,  the  slow  wave  component 
of  the  response  to  the  unaccented  word  SCHOOLBOY  is  readily  evident  in 
both  the  overt  and  covert  responses,  especially  in  electrodes  F7,  T5, 
and  C5,  and  cannot  be  attributed  to  either  eye-movement  artifacts,  jaw 
movements,  or  muscle  action  potentials.  Similar  results  were  obtained 
for  all  five  words  in  the  covert  condition. 

Figure  14  illustrates  the  stability  of  an  EEG-patterned  response  word 
w^hin  a subject  from  one  occasion  to  another  and  from  overt  to  covert 
conditions.  All  three  traces  for  _S  B are  the  smoothed  templates  for  EEG 
electrode  T5  to  the  word  TIP.  The  top  trace  is  for  overt  Session  5, 
the  middle  trace  on  overt  Session  6 two  weeks  later,  and  the  bottom  trace 
is  on  the  covert  condition  (Session  7)  six  months  after  Session  5.  The 
covert  response  is  time  justified,  and,  although  there  is  a fair  amount 
of  amplitude  variability,  the  three  responses  (between  the  cursors)  are 
almost  identical.  It  should  be  noted  that  this  is  one  of  our  better 
examples,  nevertheless,  if  classifications  are  made  between  the  covert 
responses  and  overt  templates  or  vice  versa,  the  percentage  of  correct 
responses  is  about  the  same  as  for  Within  Sessions  on  the  overt  trials. 


For  W3 , however,  analysis  on  the  CDC-6400  showed  the  percentages 
of  correct  classifications  across  all  electrodes  and  words  to  be  no  greater 
than  chance.  Perhaps  there  was  a greater  time  variation  error  for  S W3 
than  for  S B7,  which  was  amplified  by  the  correlational  time  justification 
procedure.  For  this  reason,  we  reclassified  both  S B7  and  S W3  responses 
on  the  Linc-8  after  removal  of  as  many  sources  of  error  as  possible. 
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FIGURE  14  STABILITY  OF  RESPONSE  AS  SHOWN  IN  TEMPLATES  FOR  EEG  ELECTRODE  T5 
FOR  SAME  SUBJECT  ON  DIFFERENT  SESSIONS  AND  FOR  OVERT  AND  COVERT 
CONDITIONS,  SUBJECT  B 


Error  Reduction 


To  eliminate  or  reduce  error  due  to  amplitude  variation,  each  EEG 
response  for  all  four  electrodes  and  all  words  for  both  Ss  was  smoothed 
twice  using  a five-point  triangulation  algorithm.  Since  the  EEG  response 
to  overt  and  covert  speech  production  is  a slow  wave,  the  variability  of 
the  nonresponse  components  of  the  EEG  epoch  was  reduced,  and  the  response 
itself  was  enhanced.  Second,  the  smoothed  EEG  epochs  were  plotted  in- 
dividually on  an  X-Y  plotter  to  identify  visually  the  covertly  produced 
response.  Third,  each  response  was  time  justified  by  shifting  ore  with 
respect  to  the  other  until  the  mean  square  (MS)  differences  between  the 
responses  over  the  response  period  only  was  a minimum.  This  produced  a 
reduction  of  the  variability  contributed  by  that  part  of  the  EEG  record 
not  involved  in  the  actual  response  and  an  exclusion  of  some  of  those 
portions  of  the  response  that  would  contribute  error  due  to  temporal 
variation  (expansion  or  contraction  of  the  response). 


The  results  were  dramatic,  especially  for  S B7.  Table  5 shows  the 
percentage  of  correct  classifications  for  the  initial  overt  and  covert 
calculations  in  Columns  1 and  2,  respectively,  and  for  the  covert  calcu- 
lations after  the  sources  of  error  had  been  removed  (overt  responses 
were  not  reevaluated).  Visual  examination  of  the  time-justified  records 
compared  with  the  templates  revealed  that  the  remaining  error  was  largely 
due  to  temporal  variation  and  amplitude  variation. 


Table  5 


! 

; 


E 


PERCENTAGE  OF 

CORRECT 

RESPONSE-WORD 

CLASSIFICATIONS 

FOR  EEG  FOR  OVERT  AND 

COVERT  SPEECH 

FOR  SUBJECT  B7 

before 

Error  Removal 

After  Error 

Electrodes  and 

Overt 

Covert 

Remova 1 

Combinations 

Response  ResDonse 

Covert  Response 

F7 

22% 

32  7, 

56% 

T5 

14 

18 

72 

C5 

27 

14 

68 

T6 

29 

28 

52 

All  EEG 

29 

27 

62 

Note:  All  are  significant  at  p < 0.001  by  a chi  square  test. 
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For  S_  W3,  electrode  F7,  the  Improvement  for  specific  words  was  BAN: 
10%;  PAN:  40%;  MAN:  20%;  RITE:  20%;  RIGHT:  10%;  and  WRITE:  10%. 


Therefore,  we  can  conclude  that  the  EEG  reveals  similar  responses 
during  covert  speech  (silent  reading)  to  that  during  overt  speech  and 
that  these  responses  may  be  classified  correctly  by  a computer  with  a 
fairly  high  degree  of  accuracy,  using  pattern  recognicion  techniques, 
when  the  major  sources  of  error  have  been  removed. 


Articulatory,  Semantic,  and  Contextual  Components^ 

Introduction 

Results  obtained  clearly  indicated  that  overtly  or  covertly  spoken 
words  may  be  identified  by  computer  analysis  of  electrophys iological 
patterns  correlated  in  time  with  the  speech.  Our  language  tasks  had 
involved  "discrete”  words— that  is,  words  out  of  context  that  are  seman- 
tically different  and  have  no  common  articulatory  components.  However, 
many  words  in  English  are  either  homonyms  (words  with  the  same  pronunci- 
ation but  different  meanings  when  spoken  in  context,  such  as  "write," 
right,"  and  "rite"),  or  are  very  similar,  such  as  words  varying  only  in 
the  initial  consonant  (for  example,  "ban,"  "pan,"  "man,"  and  "fan"). 

Logically,  at  least  some  of  the  biological  activity  associated  with 
pronunciation  of  these  words  is  purely  "mechanical,"  especially  in  the 
EMG.  That  is,  it  is  associated  with  articulation,  the  controlling  of 
the  vocal  musculature  for  proper  pronunciation  per  se.  On  the  other 
hand,  it  is  possible  that  some  other  components  of  the  biological  activity 
may  be  associated  with  the  meaning  of  the  word,  especially  in  the  EEG. 

If  such  components  do  exist,  a biocybernetic  communication  system  must 
be  able  to  identify  them;  if  they  do  not  exist,  some  other  form  of  dis- 
crimination  between  similarly  articulated  but  semantically  different 
words  must  be  devised. 

To  test  for  possible  discriminable  articulatory  components,  a group 
of  CVC  monosyllables,  in  which  only  the  initial  consonant  differed,  was 
used.  Since  we  also  wished  to  have  a source  of  comparison  with  the  data 
obtained  earlier,  we  used  the  same  EMG  and  EEG  electrode  recording  sites. 
Three  bi labials--B,  P,  and  M--and  the  labio-dental  F were  chosen  as  the 
initial  consonants  for  the  CVC  group,  since  maximum  EMG  responses  could 
be  recorded  from  the  lips.  The  middle  vowel  chosen  was  A,  and  the  final 
consonant  was  the  nasal  N.  Thus,  the  four  CVC  words  were  BAN,  PAN,  MAN, 
and  FAN. 

Our  assumptions  were  that  if  there  was  a nondiscriminable  articula- 
tory component,  confusion  should  be  a maximum  at  the  level  of  EMG  classi- 
fication, that  B and  p should  most  often  be  confused  with  one  another, 
since  they  differ  only  in  voicing;  that  M would  be  confused  more  with  B 


This  study  was  carried  out  as  part  of  a Master's  Thesis  by  Mrs.  Patricia 
A.  Johnson  for  the  Department  of  Psychology,  San  Francisco  State  Univer- 
sity, CA. 
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and/or  P than  F;  and  that  F would  least  likely  be  confused  with  either 
B,  P,  or  M.  That  is,  at  the  level  of  EMG  classification,  the  labio- 
dental F should  be  least  likely  to  be  confused  with  the  three  bilabials 
since  the  overt  production  of  F uses  a different  muscle  configuration 
than  that  required  for  the  two  bilabial  plosives  B and  P and  for  the 
bilabial  nasal  M.  On  the  other  hand,  if  B and  P are  not  confused,  the 
biological  activity  may  be  capable  of  discriminating  voicing  (B  is  voiced, 

P is  unvoiced).  The  F discrimination  weald  be  particularly  good  at  EMG 
electrode  site  13/16  (upper  lip)  since  labio-dental  forms  involve  the 
teeth  and  lower  lips  more  than  any  other  part  of  the  facial  musculature. 

In  addition,  if  there  is  not  a separate  articulatory  component  inte- 
to  the  EEG  pattern  associated  with  a given  word  during  overt  speech, 
the  EEG  should  be  as  confused  as  the  EMG.  However,  if  the  EEG  is  free  of 
such  mechanical  constraints,  or  has  specific  articulatory  components  and 
discriminable  contextual  components,  we  would  expect  the  EEG  to  be  the 
superior  classifier. 

In  the  case  of  the  homonyms,  the  phonetic  form  Rit  (no  semantic  com- 
ponent), and  three  of  its  six  possible  semantic  forms  (WRITE,  RIGHT,  and 
RITE)  were  chosen  as  stimuli.  The  purpose  of  this  condition  was  to  equate 
any  possible  articulatory  component,  since  all  four  words  should  be  pro- 

Inounced  exactly  the  same  (with  possible  minor  variations)  by  a given 

subject.  As  with  the  CVC  words,  there  should  be  a high  degree  of  confusion 
in  the  EMG  classification  among  the  "Rit"  forms.  For  the  EEG,  there 
should  be  a high  degree  of  correct  classification  if  a semantic  component 
exists  in  the  EEG.  If,  however,  similar  levels  of  confusion  exist  in 
both  EMG  and  EEG  classification  of  these  words,  or  if  the  EMG  remains 
the  superior  classifier,  we  may  make  the  following  assumptions:  The 
variability  in  physiological  response  patterns  at  our  present  level  of 
technology  is  too  great  to  allow  for  precise  classification;  we  may  not 
be  measuring  from  the  correct  EEG  locations;  or  we  need  to  determine  the 
interrelationships  among  patterns  from  some  combination  of  EEG  sites. 

Finally,  each  CVC  word  and  each  of  the  three  semantic  forms  of  the 
homonym  "Rit"  were  put  into  an  identical  phrase  context.  The  purpose  of 
this  phrasing  was  to  identify  the  effects  of  preceding  and  subsequent 
words  on  the  CVC  words  and  homonyms  of  interest  and  to  determine  whether 
the  computer  could  classify  a word  within  a phrase  context.  Our  assump- 
tions were  that  if  no  discriminable  EMG  articulatory  component  were 
present,  there  would  be  a high  degree  of  confusion  among  CVC  phrases 
and  possibly  between  a CVC  word  and  that  same  CVC's  phrase.  Similar 
assumptions  were  made  in  terms  of  the  homonyms  and  homonym  phrases. 
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Subjects  and  Apparatus 

Subjects  were  five  adult,  right-handed  human  volunteers,  two  males 
and  three  females,  ages  18  to  38  years,  henceforth  designated  as  U,  V, 

W,  Y,  and  Z.  Each  subject  participated  in  at  least  two  experimental 
sessions  and  two  of  the  Ss  (W,  Y)  participated  in  three  or  more  sessions, 
designated  1,  2,  3....  Duration  of  each  session  ranged  from  1-1/2  to 
3 hours,  not  including  pre-  and  post-session  calibration  procedures. 
Before  each  of  the  first  two  sessions,  all  Ss  were  given  a standard 
dichotic  1 '•-'n’Vg  test  (see  Part  II  on  Hemispheric  Differences).  Elec- 

trode location  and  equipment  setup  were  similar  to  the  descriptions 
given  in  the  section  on  methods. 


Language  Task 

i 

The  14  words  and  phrases  used  are  shown  in  Table  6.  As  a control 
to  ensure  that  language  production  is  different  from  simple  mechanical 
movement,  Ss  also  were  presented  with  three  control  stimuli.  The  first 
control  required  S to  make  the  motor  response  of  pressing  a key  to  the 
stimulus,  "PRESS  KEY";  a second  control  required  a swallowing  response 
to  the  stimulus  word  "SWALLOW";  and  a third  control  required  a verbal 
response  to  the  phonetic  stimulus  "Rit"  (no  semantic  component). 


Table  6 


LIST  OF  WORDS,  PHRASES,  AND  CONTROLS  USED 
FOR  TESTING  ARTICULATORY  AND  SEMANTIC 


cvc 

! Words 

COMPONENTS 
Monosyl labic 

and 

Phrases 

Words/phrases 

Controls 

BAN 

WRITE 

PRESS  KEY 

PAN 

RITE 

SWALLOW 

MAN 

RIGHT 

RIT 

FAN 

THE 
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Procedure 

The  section  on  Methods  details  the  procedures  used.  Briefly,  after 
electrodes  were  attached,  Ss  were  comfortably  seated  in  the  semi-dark, 
electrically  shielded  isolation  booth.  All  electrodes  then  were  attached 
to  the  junction  box  leading  to  the  Beckman  Dynograph.  When  all  electrodes 
were  verified  as  operating  properly,  the  S was  carefully  instructed  as 
to  the  general  procedure  of  the  experiment  and  the  importance  of  minimum 
eye,  facial,  body,  or  swallowing  movements  during  a trial  interval.  A 
recording  session  was  then  begun. 

At  the  sound  of  the  first  click  in  the  headset,  S was  to  attend  to 
a green  fixation  point  (5-mm  diameter)  located  in  the  center  of  the  rear- 
projection  screen.  One  second  later,  the  stimulus  was  presented  on  the 
screen,  at  which  time  S was  to  continue  attending  to  the  stimulus  word 
and  to  be  prepared  to  respond  as  soon  as  the  second  click  sounded.  Ss 
were  requested  to  spe.  k their  responses  distinctly  and  normally  into 
the  microphone.  When  a 5-sec  trial  interval  was  completed,  the  projector 
light  illuminating  the  stimulus  was  automatically  extinguished,  and  S 
was  instructed  to  relax  and  await  the  beginning  of  the  next  trial. 

Slides  were  changed  manually  during  the  intervals  between  each  of 
the  17  stimulus  presentations.  A single,  random  presentation  of  each  of 
the  17  stimuli  (7  words,  7 phrases,  and  3 controls;  see  Table  6)  consti- 
tuted one  set  of  trials.  Following  the  presentation  of  one  complete  set, 
the  17  slides  were  shuffled  for  raudom  presentation.  A second  presenta- 
tion of  the  same  17  stimuli  was  conducted  in  a manner  identical  to  that 
described  above.  This  procedure  was  followed  for  a total  of  ten  trial 
sets.  Thus,  for  each  experimental  session,  which  consisted  of  the  ten 
trial  sets  of  17  stimuli  each,  data  were  accumulated  for  170  stimulus 
presentations  (70  words,  70  phrases,  and  30  controls).  Each  subject 
participated  in  at  least  two  experimental  sessions.  Intervals  between 
sessions  were  typically  two  weeks  (range--one  week  to  one  month). 

Following  each  session,  the  was  given  a standard  dichotic  listening 
test. 


Data  Collection  and  Analyses 

A data  block  comprised  the  EMG  and  EEG  data  collected  during  a 5-sec 
trial  interval.  As  described  in  "Methods,"  analog  EMG  and  EEG  data  ob- 
tained during  the  language  tasks  were  stored  on-line  by  the  Linc-8  com- 
puter. Thus,  all  170  verbal  and  control  responses  for  a given  S for  a 
given  session,  as  well  as  a set  of  20  calibration  signals,  were  stored 
on  Line  tape.  These  data  then  were  transferred  to  the  Pertec  digital 
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tape  recorder,  unpacked  by  the  CDC-6400  computer,  and  the  data  words, 
phrases,  and  controls  were  reordered  in  sequence  for  processing.  Data 
were  then  time  justified  using  a correlational  technique,  and  all  re- 
sponses were  classified. 


Response  classifications  were  made  for  each  of  the  170  stimuli  per 
subject  per  session  (Within  Subjects/Within  Sessions  classification). 

As  in  the  earlier  data  analysis,  in  addition  to  the  response  classifica- 
tions for  each  of  the  six  electrodes,  we  decided  to  combine  (1)  the  two 
EMG  responses,  (2)  the  four  EEG  responses,  and  (3)  the  six  EMG  and  EEG 
responses  so  as  to  assess  the  relative  contribution  of  types  and  number 
of  electrode  responses  to  the  computer  classification. 


Results 


General 


EEG  and  EMG  templates  for  the  three  CVC  bilabials  BAN,  PAN, 
and  MAN,  and  the  labio-dental  FAN  are  illustrated  for  the  second  session 
of  S Y in  Figures  15  and  16.  It  is  evident  that  the  two  EMG  channels, 
although  different  from  each  other,  are  practically  identical  within 
each  electrode  among  the  three  bilabial  words  BAN,  PAN,  and  MAN.  There 
are  equally  strong  similarities  for  each  EEG  electrode  among  these  words 
during  the  response  period.  However,  note  the  very  definite  differences 
in  both  the  EMG  and  EEG  patterns  for  the  labio-dental  word  FAN.  Thus, 
on  visual  inspection  at  least,  the  patterns  of  muscle  contraction  and 
the  correlated  brain  activity  for  words  with  very  similar  articulation 
of  the  first  sound  apparently  are  nondiscriminable,  whereas  a similar- 
sounding word  with  different  articulation  of  the  first  sound  is  discrim- 
inable  at  both  the  muscle  and  brain  activity  levels.  The  result  of  artic- 
ulatory similarity  being  reflected  in  both  the  EMG  and  EEG  of  words 
sounding  alike  also  was  found  for  the  four  "Rif  forms,  as  shown  in 
Figures  17  and  18. 


Within  Subjects 

Table  7 summarizes  the  percentage  of  correct  response  classi- 
fications for  each  channel  and  combination  for  both  sessions  for  all 
subjects.  The  percentage  of  correct  classifications  ranged  from  7 to 
62%,  with  the  combination  of  all  electrodes  being  the  best  classifier 
in  almost  all  cases.  We  may  conclude  the  following  from  Table  7: 
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FIGURE  16  TEMPLATES  FOR  ALL  ELECTRODES  FOR  CVC  WORDS  MAN  (a)  AND  FAN  (b),  SUBJECT  Y2 


Table  7 


PERCENTAGE  OF  CORRECT  RESPONSES  FOR  ALL  SUBJECTS  AND  SESSIONS, 
WITHIN  SESSION  CLASSIFICATION,  SAMPLE  POINTS  104-210 


Channel 

Ul 

U2 

VI 

V2 

W1 

W2 

Y2* 

Y3 

Z1 

72 

13/16 

1 

17 

35 

34 

40 

48 

43 

22 

35 

29 

Ct  4 

34 

7/8 

2 

25 

27 

32 

36 

44 

49 

20 

34 

25 

26 

F7 

3 

20 

25 

34 

26 

47 

53 

36 

24 

13 

33 

T5 

4 

24 

19 

25 

24 

36 

47 

28 

27 

22 

36 

C5 

5 

24 

24 

24 

19 

38 

51 

39 

24 

18 

32 

T6 

6 

19 

24 

26 

7 

43 

53 

27 

30 

21 

37 

All 

EMG 

32 

36 

40 

43 

49 

52 

25 

41 

32 

38 

All 

EEG 

25 

35 

37 

17 

49 

60 

41 

37 

22 

39 

All 

e lectrodes 

34 

46 

55 

28 

61 

62 

48 

45 

35 

48 

+ 

Y1  was  a preliminary  experiment  slightly  different  from  the 
paradigm  reported  here  and  therefore  is  not  included. 


(1)  For  four  of  the  five  Ss,  a greater  number  of  responses 
were  correctly  classified  in  the  second  session,  indi- 
cating a possible  practice  effect  in  some  instances 
and  a possible  fatigue  effect  in  others.  (A  similar 
trend  can  be  observed  within  a recording  session  in 
that  correct  classifications  tend  to  occur  during 
midsession,  indicating  an  initial  learning  or  practice 
effect,  followed  by  a fatigue  effect  toward  the  end  of 
the  session.) 

(2)  In  two  of  the  Ss  (W  and  Y) , the  EEG  was  equal  or  su- 
perior to  the  EMG  as  a classifier.  In  fact,  for  S W2, 
all  but  one  of  the  individual  EEG  responses  were 
better  classifiers  than  the  combined  EMG  responses. 
However,  for  the  majority  of  Ss,  the  EMG  was  either 
equivalent  to  or  better  than  the  EEG  as  a classifier. 

(3)  As  mentioned  above,  the  range  of  correct  classifica- 
tions across  Ss  was  from  7 to  627,.  This  range  is 
lower  than  that  for  the  earlier  data,  presumably  be- 
cause of  the  confusion  resulting  from  the  similarity 
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among  the  CVC  words  and  the  "Rif  forms  (as  with  the 
bisyl labic  words  in  the  earlier  data). 

(4)  Comparisons  among  subjects  indicates  that  responses 
from  S W were  classified  correctly  more  often  than 
those  from  any  of  the  others  in  this  group  of  subjects. 

When  the  data  from  all  subjects  were  grouped  together  the  re- 
suits  were  essentially  as  stated  above.  That  is,  the  two  EMG  ctanll 
orrectly  classified  32  to  42*4  of  the  responses  correctly  for  all  sub- 

io^ef fthEEG  ”n8ed  fr°m  29  t0  32'4'  F°r  £he  tuo  EMG 

four  FFr'  b °f  , resp°nses  CQrrectly  classified,  377.  for  the 

four  EEC  channels  taken  together,  and  46 % for  all  six  electrodes. 


Confusion  Matrix 


m 


cation  of  mwv  m ***  am°Unt  °f  confusion  between  computer  classifi- 

f f°.  Uke  yesPonse -words  (CVC  words,  homonyms,  and  phrases),  sets 

of  r ITrLCes  were  instructed  for  each  electrode  and  combinations 

electrodes  for  each  S on  each  session.  Table  8 shows  two  summary 
trices  for  all  Ss  and  session  for  the  period  of  the  actual  response 
(computer  samples  104  to  210).  In  Table  8(a),  the  confusion  for  both 

EEC  elect  ^ t0gether  iS  ShoWn>  and  the  confusion  for  all  four 

lectrodes  taken  together  is  shown  in  Table  8(b).  The  diagonals 

om  upper  ie  t to  lower  right  give  the  number  of  correct  classifications 

out  of  possible  100  (and  thus  the  percentage  of  correct  classifications) 

and  the  outlined  confusion  blocks  illustrate  the  confusion  between  like  ’ 

words.  Conclusions  from  Table  8 may  be  summarized  as  follows- 


(1)  In  both  the  EMG  and  EEG  responses,  the  major  confusion 
existed  only  between  like  forms,  whether  they  be  CVC 
words,  homonyms,  CVC  or  homonym  phrases,  or  mechanical 
controls.  Thus,  individual  response-words  were  success- 

ully  discriminated  from  phrases  containing  that  word, 
from  words  or  phrases  of  discretely  different  articula- 
tory production,  and  from  the  mechanical  control  responses 

(2)  The  control  responses  PRESS  KEY  and  SWALLOW  were  not 
confused  with  each  other  or  with  any  language  task, 
showing  that  the  language  components  of  each  response 
were  discriminable  as  such. 

(3)  The  mechanically  similar  bilabials  BAN,  PAN,  and  MAN 
were  confused  with  each  other  in  both  the  EMG  and  EEG 
from  one-tenth  to  one-third  of  the  time.  On  the 
other  hand,  the  mechanically  different  labio-dental 
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Table  8 


SUMMARY  CONFUSION  MATRICES  FOR 
ALL  SUBJECTS  AND  SESSIONS,  EMr-  AND  EEG  RESPONSES 

(It  EMG  RESPONSES 


1 HAN 
? PAN 
1 MAN 
A FAN 

5 PIT 

6 WPT 

7 PTE 

8 WGT 

9 THU 

10  TPO 

11  Tmo 

12  TFO 
11  T MO 

14  THU 

15  T GO 
1*  KFT 
17  SWL 


PETS  CLASSIFIED 
12  3 4 b b~ 

^1^22  19  T|  1 1 

19x^2^24  3 i d 

19  18'.?7\a  3 2 

1 0 1 

llo  7 2V 1 4 
1 1 0 5 Ib^Bx 

1 1 1 1 21  ib^ 

2 1 0 7 [23  22 

0 2 4 10  1 

3 3 5ool 

? 1 2 o 0 >J 

0 0 0 22  2 1 

0 0ol5b 

0 U 1 1 2 J 

0 1 1 4 H <♦ 

0 u 0 0 0 0 

110  4 14 


_« 3 

21  16 
2 0 14 

25s'  l 7 
19^14" 
0 0- 


b 5 
9 6 


IF  IT  WF-HE  - 
9 10  11  12  13 

b 2 3 0 0 

* 2 3 0 2 

b b 4 1 ? 

0 0 0 24  1 

0 0 1 0 0 


(b)  EEG  RESPONSES 


> GETS  CLASSIFIED  AS 

1 2 3 4 3 b 7 8 

^<►^20  1 5 9 J 2 2 0 

23^0^7  H 5 0 4 n 

20  H^49\5  4 J 2 3 

-LI ? 7^36'  - 2 0 3 4 

2 2 2 B 56  23 

1 0 3 1 1 6x2b'-J4  13 

3 0 2 0 17  23vi8'-26 

1 2 3 n |2()  25  30x1^ 

2 2 2 1 1 I 0 O' 

2lo3o2oi 

2*120001 

3 1 2 2 0 0 1 0 

2322q03i 

2 0 1 3 2 2 4 0 

1 1 1 0 0 2 1 3 

1 2 2 2 3 1 0 1 

3 1 5 1 3 3 2 2 


'4* 

"27 

3n 

0 

22- 

x*  3s- 1 8 

0 

33 

9x 

s3b\2 

0 

0 

4. 

0 

0 

4 

1 

0 

R 

4 

i 

2 

7 

4 

0 

0 

0 

0 

1 

1 

1 

0 

IF 

i r 

"EWE 

9 

10 

1 1 

12 

3 

4 

0 

3 

0 

1 

0 

3 

1 

1 

0 

1 

2 

4 

3 

7 

u 

0 

0 

1 

1 

1 

0 

1 

0 

1 

0 

1 

■xj 

1 

0 

0 

2 0 0 0 

* 2 0 0 
0 12  3 

2 2 0 3 

* 2 0 1 

5 3 2 2 

9 1 0 0 

2 4 0 0 

1*00 
10  b 0 0 

* 3 0 3 

'1*  111  0 0 
^3^33  1 l 

25^2  r ^ 1 0 

0 ?^b\4 

0 2 1 3x67 


2^22  21  9~|  2 

2«v^5\6  15  l 
lj  1*0^12  3 

Il7  15  19^33-0 

2 1 4 5-  4^ 

3 1 1 2 11> 

3 1 1 5 1J 

1 5 0 2 4 

1 0 3 0 5 


1*  15  lb  i7 
0 2 3 o 

113  1 

10  11 

0 0 5 0 

2 10  0 
2 10  0 
5 1 1 0 

1 0 0 0 
&220 
3 2 1o 

5 3 0 0 

2 3 0 1 

12  111  3 0 

.33-35  0 0 

34x 34~  s? 0 

~l  ?^6\4| 

6 1 iv 


FAN  was  discriminated  successfully  from  the  other 
three  similar  sounding  CVC  words,  especially  in 
the  EMG. 


f 


; 

■ 


f 


(4) 


Voiced  words  could  not  be  distinguished  reliably 
from  like-sounding,  unvoiced  words  (as  between  BAN 
and  PAN)  in  either  the  EMG  or  EEG,  whether  the  word 
was  isolated  or  imbedded  in  a phrase.  However,  this 
may  be  due  to  the  correlational  time- justification 
procedure,  since  EMG  activity  for  unvoiced  words 
occurred  later  in  time  than  for  voiced  words,  and, 
thus,  appeared  to  the  computer  as  a temporal  shift. 
Similarly,  nasal  words  were  not  distinguished  re- 
liably from  plosive  like-sounding  words  (as  between 
MAN  and  PAN  or  BAN),  whether  as  an  isolated  word  or 
when  imbedded  in  a phrase. 


[ 

l 

T 


To  evaluate  the  relative  differences  between  the  EMG  and  EEG 
for  similarly  articulated  words  and  phrases  (CVC  words),  and  for  identi- 
cally pronounced  words  and  phrases  (homonyms),  a table  was  constructed 
(Table  9)  of  the  ratios  of  the  number  of  correctly  classified  responses 
to  the  number  of  confused  responses  within  each  confusion  block  of 
Table  8.  Results  of  tests  of  the  significance  of  the  differences  between 
the  various  ratios  of  Table  9 (i.e.,  between  the  correlated  proportions) 
are  shown  in  Table  10.  The  upper  portion  of  Table  10  compares  different 
confusion  blocks  for  both  the  EMG  and  EEG,  and  the  lower  portion  compares 


Table  9 

RATIOS  OF  NUMBER  OF  CORRECTLY  CLASSIFIED 
RESPONSES  TO  NUMBER  OF  CONFUSED  RESPONSES 
WITHIN  CONFUSION  BLOCKS  FOR  EMG  AND  EEG 


Confusion  Block 

Both  EMG 

All  EEG 

CVC  words 

1. 16 

1.09 

Homonyms 

0.45 

0.30 

CVC  phrases 

1.08 

0.67 

Homonym  phrases 

0.88 

1.02 

Mechanical  controls 

9.59 

33.00 

A 
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Table  10 

NORMAL  CURVE  DEVIATIONS  (Z)  FOR  THE  SIGNIFICANCE  OF 
THE  DIFFERENCES  BETWEEN  VARIOUS  RATIOS  IN  TABLE  9* 


Confusion  Block  Ratios  Comoared 

Z- 

EMG 

Scores 

EEG 

CVC  words  versus  homonyms 

5.73 

7.42 

CVC  words  versus  CVC  phrases 

0.37 

3.07 

Homonyms  versus  homonym  phrases 

4.80 

8.01 

CVC  phrases  versus  homonym  phrases 

1.08 

2.80 

Confusion  Block  EMG 

versus  EEG 

CVC  words  0.33 
Homonyms  2.74 
CVC  phrases  3.03 
Homonym  phrases  0.  68 


For  Z < 1.96,  differences  not  significant. 
For  1.96  < Z < 2.58,  p < 0.05. 

For  Z > 2.58,  p < 0.01. 


the  EMG  with  the  EEG  for  the  four  confusion  blocks.  The  following  may 
be  concluded  from  Tables  9 and  10: 


(1)  For  both  the  EMG  and  EEG,  CVC  words  were  significantly 
less  confused  (p  < 0.01)  than  homonyms  (CVC  words  versus 
Homonyms  in  Table  10).  Thus,  at  both  the  vocal  muscula- 
ture and  brain  activity  levels,  isolated  words  only 
"sounding"  alike  may  be  discriminated,  whereas  those 
actually  pronounced  alike  may  not.  Thus,  discriminable 
articulatory  or  semantic  components  probably  exist  for 
like-sounding,  but  differently  pronounced  words,  whereas 
only  a single  articulatory  component  exists  for  words 
pronounced  alike. 

(2)  The  existence  of  a separate  semantic  component  for  CVC 
words  in  the  EEG  is  suggested  by  comparing  the  isolated 
words  with  those  contained  in  the  contextual  phrase 
(CVC  Words  versus  CVC  Phrases  in  Table  10,  p < 0.01). 
Thus,  the  EEG  is  capable  of  discriminating  similarly 

63 


articulated  words  in  context  when  there  is  no  significant 
difference  in  the  actual  articulation  of  these  words  in 
context  (contrast  with  EMG). 

(3)  Even  homonyms  are  more  likely  to  be  classified  correctly 
if  they  are  contained  in  a contextual  phrase  than  when 
isolated  (Homonyms  versus  Homonym  Phrases  in  Table  10). 
This  difference  is  significant  at  p < 0.01  for  both  EMG 
and  EEG,  but  especially  for  the  EEG.  This  suggests  that 
Ss  can  evoke  only  distinctive  semantic  aspects  of  a 
lexical  item  with  only  one  pronunciation  but  can  evoke 
several  semantic  interpretations  when  that  item  is  placed 
in  a context. 

(4)  Whereas,  for  the  EMG,  CVC  words  are  less  confused  than 
homonyms,  CVC  phrases  are  only  slightly  less  confused 
than  homonym  phrases  (CVC  Phrases  versus  Homonym  Phrases 
in  Table  10),  but  not  significantly  so.  For  the  EEG, 
however,  CVC  phrases  are  significantly  more  confused 
than  homonyms  (p  < 0.01).  Again,  this  may  reflect  the 
differences  in  correct  classification  when  words  are 
used  in  context  rather  than  when  they  are  isolated, 
with  identically  pronounced  words  being  more  discrim- 
inable  than  "like-sounding"  words. 

(3)  Over  all,  EMGs  tend  to  be  less  confused  than  EEGs  (lower 
portion  of  Table  10),  but  significantly  so  only  for 
homonyms  and  CVC  phrasi  s (p  < 0.01).  We  can  account  for 
this  only  in  terms  of  higher  EEG  variability.  In  Table  9, 
the  EEG  does  have  a higher  percentage  of  correct  classi- 
fications for  homonyms  than  the  EMG,  but  this  difference 
is  not  significant. 

(6)  The  mechanical  controls  are  completely  discriminated  from 
all  language  tasks  and  vice  versa  at  p < 0.0001.  There- 
fore, we  can  conclude  that  all  results  with  the  language 
tasks  were  related  to  language  function  and  not  to  any 
confusion  from  mechanical  interference. 


On-Line  Analysis 
Introduction 


The  results  from  the  off-line  computer  analysis  of  the  EEG  coincident 
with  overt  and  covert  speech  described  in  the  preceding  sections  were 
sufficiently  above  chance  expectation  (on  the  average  statistically 
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significant  at  p < 0.001)  to  conclude  that  a biocybemetic  communication 
system  as  originally  proposed  is  feasible.  However,  to  be  useful,  such 
a communication  system  should  be  employable  in  real-time,  with  on-line 
communication  between  the  brain  of  a man  and  a computer.  This  last 
section  of  Part  I describes  our  efforts  in  this  direction. 

On-line  analysis  was  carried  out  over  a period  of  1-1/2  years  in  a 
generally  dynamic  fashion.  That  is,  no  single  set  of  experiments  was 
conducted  with  a large  number  of  subjects  to  investigate  a given  param- 
eter. Rather,  a basic  paradigm  was  formulated  for  the  on-line  analysis 
and  then  variations  were  made  in  this  paradigm  for  exploration  of  several 
important  features.  For  example,  single  electrode  responses  were  com- 
pared with  responses  averaged  across  several  electrodes;  various  EEG 
recording  sites  were  compared  for  maximum  correct  computer  classification; 
left  hemisphere  electrode  responses  were  averaged  together  and  compared 
with  averaged  right  hemisphere  electrode  responses,  and  both  sets  of 
responses  were  compared  with  the  average  of  electrodes  from  both  hemi- 
spheres taken  together;  several  types  of  computer  algrorithms  were  com- 
pared with  each  other,  and  attempts  were  made  to  correct  for  temporal 
variation  by  computer  modification  so  as  to  improve  correct  classification 
rates;  different  amounts  of  EEG  data  were  compared  (i.e.,  larger  or 
smaller  epochs),  as  well  as  different  periods  (i..e.,  EEG  data  collected 
during  and  after  the  actual  response  of  S to  the  stimulus  word);  trained 
subjects  were  compared  to  naive  subjects;  and  repetitive  (fixed-serial- 
order)  presentations  of  stimuli  were  compared  with  random  presentations. 
For  most  of  these  parameters,  overt  speech  was  compared  with  cc  r~rt 
speech. 

Because  of  the  volume  of  results  obtained  from  this  procedure  (de- 
scribed in  detail  in  Semi-Annual  Technical  Progress  Report  Nos.  4,  5, 
and  6),  this  section  is  limited  to  a description  of  the  basic  paradigm 
and  summarizes  the  results  of  the  various  comparisons  described  above. 

A concluding  statement  is  given  at  the  end  of  this  section  regarding 
the  most  feasible  on-line  technique  as  determined  from  our  results. 


Methods  and  Procedures 

A total  of  31  Ss  were  run  on  the  on-line  experiments  for  a total  of 
140  experimental  sessions.  Ss  were  both  male  and  female  volunteers, 
ranging  in  age  from  21  to  50  years.  Sessions  lasted  from  two  to  six 
hours  (mean  of  three  hours  each). 


One  major  set  of  seven  words  was  used,  with  minor  variations.  In 
these  experiments,  we  were  preparing  for  a demonstration  of  our  procedures 
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as  well  as  assessing  and  improving  our  computer  techniques  for  on-line 
classification.  Therefore,  words  were  chosen  on  the  basis  of  two  practi- 
cal demonstrations.  The  first  set  of  words  was  chosen  in  conjunction  with 
a demonstration  for  EEG-computer  control  of  an  in- lab  video  camera  used 
to  observe  a free-living  group  of  adolescent  chimpanzees.  The  seven 
words  in  this  instance  were  chosen  on  the  basis  of  the  video  camera 
capability,  including  movement  to  the  right,  left,  up,  and  down,  and 
movement  via  the  zoom  lens  to  far  and  near.  Thus,  the  words  to  be 
classified  by  the  computer  on  the  basis  of  the  EEG  responses  were  RIGHT, 
LEFT,  UP,  DOWN,  NEAR,  and  FAR.  The  seventh  word,  S.OP,  was  used  in 
interaction  with  the  other  six  words  in  controlling  ^.he  video  camera. 

For  an  alternate  demonstration,  the  movement  of  a dot  on  a video  monitor 
to  the  right,  left,  up,  or  down,  as  well  as  the  fast  or  slow  movement 
in  any  of  the  directions,  was  chosen.  Thus,  the  words  used  were  RIGHT, 
LEFT,  UP,  DOWN,  FAST,  and  SLOW.  Again,  the  seventh  word  was  STOP. 

Details  of  the  general  procedures  for  electrode  preparation  and 
attachment  were  given  earlier.  However,  certain  changes  were  made  in 
the  experimental  design  for  these  studies,  including  elimination  of  the 
EMG,  and  recording  of  the  EEG  from  eight  electrodes  (four  over  each  hemi- 
sphere from  homologous  leads;  F7,  C5,  T3,  and  Ti  for  the  left  hemisphere, 
and  F8,  C6,  T4,  and  T6  for  the  right  hemisphere;  all  EEG  leads  were 
referenced  to  the  vertex). 

After  electrodes  were  attached,  the  S was  seated  in  a semidark, 
electrically  shielded  booth,  and  the  electrodes  were  plugged  into  a 
junction  box  attached  to  the  recording  equipment.  When  all  electrodes 
were  verified  for  adequate  resistance  (typically  under  lK-ohm)  and  op- 
timal electrophysiological  recording,  an  experimental  session  was  begun. 
The  session  consisted  of  a visual  presentation  (on  a Conrac  video 
monitor)  of  the  set  of  seven  words  once  every  5 sec  for  periods  of  1 sec 
each.  Stimulus  cues  and  presentation,  as  well  as  onset  of  data  collec- 
tion, analysis,  classification,  and  feedback,  were  controlled  by  the 
DEC  Linc-8  computer. 

When  a IS  was  ready,  the  computer  automatically  initiated  a template- 
building phase  of  the  experiment.  One  second  after  an  initial  warning 
click  in  the  earphones,  a word  was  automatically  presented  on  the  video 
screen  for  1 sec.  The  was  instructed  to  attend  to  the  video  screen 
at  the  presentation  of  the  warning  click  and  to  read  the  word  aloud 
(overt  condition)  or  read  the  word  silently  (covert  condition)  as  soon 
as  the  word  appeared  on  the  screen.  Data  were  collected  only  during  the 
1 sec  that  the  stimulus  was  present  on  the  screen,  commencing  180  msec 
after  the  onset  of  the  word  and  ending  180  msec  before  the  word  ended. 

At  a sampling  rate  of  100/sec,  this  resulted  in  64^  samples  per  trial 
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per  electrode.  After  the  first  presentation  of  each  word  (i.e.,  on  the 
second  and  subsequent  presentations),  the  computer  would  present  the 
classification  results  to  the  S after  each  trial,  until  all  words  had 
een  presented  ten  times  each  (70  presentations  in  all).  This  usually 
constituted  an  experimental  session. 


Results 


General 


Table  11  presents  a typical  on-line  classification  data  sheet 
in  this  case  for  S 3,  Session  B,  overt  condition.  Similar  data  sheets  ’ 
were  completed  for  each  subject  for  each  condition  per  experimental 
session.  Across  the  top  and  down  the  left  side  of  the  data  sheet  are 
listed  the  seven  stimulus  words.  The  49  individual  boxes  were  used  for 
entry  of  data,  i.e.,  which  repetition  of  the  stimulus  word  (to  the  left) 
was  classified  as  the  word  printed  above  each  column.  For  example,  the 
third  repetition  of  the  word  RIGHT  was  classified  as  FAR,  whereas  the 
fourth  and  seventh  repetitions  were  classified  as  STOP.  Thus,  the  seven 
boxes  along  the  diagonal  (within  word  boxes)  list  the  correct  classifi- 
cation instances  for  each  of  the  seven  words.  The  seven  rows  and  columns 
(minus  the  within  word  boxes)  indicate  the  incorrect  classification  in- 
stances (i.e.,  confusions)  for  each  word  across  the  horizontal  axis  with 
the  word  to  the  left. 


In  addition,  the  column  on  the  far  right  side  of  Table  11  gives 
correct  classification  p_ercentages  for  each  individual  word,  as  well  as 
the  overall  percentage  of  correct  classification  (in  this  instance  44%). 
Note  that,  for  this  S,  in  this  condition,  UP  was  the  word  most  often 
classified  correctly  (70%);  it  was  followed  closely  by  LEFT  (60%)  and 
by  RIGHT  and  FAR  (50%),  but  DOW  was  never  classified  correctly.’  Note 
also  the  general  tendency  for  correct  classifications  to  be  grouped 
(such  as  8,  9,  10  for  RIGHT;  4,  5,  6,  7 for  LEFT  and  so  on).  This  ten- 
dency was  observed  consistently  in  all  Ss  and  in  all  sessions.  Finally 
Table  11  shows  that  correct  classifications  are  more  likely  to  occur 
after  the  first  three  or  four  presentations.  This  tendency  may  indicate 
a practice"  effect,  whereas  another  consistently  observed  effect,  a 
slight  decrease  in  correct  classification  during  the  last  few  presenta- 
tions, may  indicate  a "fatigue"  effect.  This  apparent  fatigue  effect 
also  is  observed  during  extended  sessions;  general  classification  per- 
centages begin  to  decrease  after  the  first  four  or  five  presentations 
in  a single  r =>ssion. 
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Table  11 

TYPICAL  SHEET  FOR  ON-LINE  ANALYSES 


Percentage 


RIGHT 

LEFT 

UP 

DOWN 

NEAR 

FAR 

STOP 

Correct 

RIGHT 

1,5,8, 

9,10 

2,6 

3 

4,7 

50% 

LEFT 

8 

4,5,6, 

7,9,10 

1,3 

2 

60 

2,4,5, 

1 

3,8 

UP 

6,7,9, 

10 

70 

DOWN 

6 

2,7, 

10 

1,3 

4,5,8, 

9 

0 

NEAR 

9 

5 

OO 

- O 

3 

1,2,6 

40 

FAR 

1,4 

6 

7,9 

2,3,5, 

8,10 

50 

STOP 

7 

9 

3,6,10 

1 

2,4,5, 

8 

40 

Total  447» 


Condition:  Right  hemisphere — F8,  C6,  T4,  T6,  02 — 

averaged/update  x 5 


Subject:  3B 

Date:  1/25/74 


Initial  On-Line  Results 


To  compare  the  results  of  the  various  on-line  classification 
procedures  of  EEG,  we  developed  a new  format  of  data  presentation.  This 
format  is  illustrated  in  Table  12,  which  presents  results  for  seven  Ss 
from  the  initial  on-line  experiments  for  both  overt  and  covert  speech 
over  a number  of  sessions  for  each  subject.  The  results  are  divided 
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further  into  groups  that  compare  the  "best  word"  percentage  of  correct 
classification  for  the  overt  and  covert  conditions  with  the  "over  all 
words"  percentage  of  correct  classification.  This  division  is  important 
because  averaging  correct  classification  percentages  across  all  words 
tends  to  smear  results  obtained  on  individual  words,  where  the  nercent- 
ages  of  correct  classifications  are  often  very  high.  Such  high  classifi- 
cation rates  on  individual  words  is  indicative  of  the  feasibility  of 
improving  the  overall  correct  classification  rate,  since  these  high 
classification  rates  cannot  have  occurred  by  chance.  Thus,  if  it  can  be 
determined  why  some  words  are  so  often  correctly  classified  while  others 

are  not,  a method  for  improving  the  classification  over  all  words  may  be 
suggested. 


To  illustrate  the  variability  in  correct  classification  rates, 
the  range  of  percentage  of  correct  responses  over  the  number  of  sessions 
is  given,  as  well  as  the  average  percentages  across  sessions,  for  each 

S on  both  the  overt  and  covert  conditions  for  the  best  word  and  over  all 
wo  rd  s . 


Data  for  the  results  presented  in  Table  12  were  collected  and 
analyzed  as  follows;  EEG  recordings  were  obtained  from  five  electrodes 
placed  over  both  hemispheres.  The  1-sec  epoch  of  EEG  from  each  electrode 
for  each  word  presentation  was  first  smoothed,  using  a five-point  trian- 
gular smoothing  routine,  and  the  analog  signals  for  all  five  electrodes 
then  were  averaged  together  for  each  sample  point  over  the  200  computer 
samples  for  a single  average  response  for  each  word.  The  average  response 
for  the  first  presentation  of  each  word  represented  the  initial  template. 


To  classify  the  average  responses  for  the  nine  succeeding  woid 
presentations,  Mean  Square  (MS)  Difference  between  a given  input  average 
response  and  the  existing  seven  templates  (one  for  each  word)  was  com- 
puted, and  the  input  response  then  was  classified  as  that  word  for  which 
the  MS  difference  was  a minimum.  After  each  classification,  the  appro- 
priate template  was  updated  by  a simple  averaging  of  the  existing  template 
and  the  input  averaged  response.  The  updated  templates  then  were  used 
for  classification  of  the  input  averaged  responses  for  the  next  series 
of  word  presentations. 


Results  in  Table  12  show  a high  degree  of  variability  between 
Ss,  words,  and  conditions.  For  example,  the  range  of  percentage  of 
correct  classifications  for  the  best  word  in  the  overt  condition  is  from 
10  to  100%  across  all  Ss,  whereas  it  is  only  20  to  60%  for  the  covert 
condition.  All  Ss  except  GL  had  correct  classification  rates  of  70%,  or 
higher  (PJ  as  much  as  100%)  on  individual  words.  Even  the  average  per- 
centages of  correct  responses  for  individual  best  words  were  relatively 
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high,  all  significant  at  p <0.001,  except  for  Ss  MT  and  GL,  which  were 
significant  at  0.01  <p  <0.05.  Thus,  the  results  show  that  high  perfor- 
mance may  be  expected  of  most  £3s  on  some  words  most  of  the  time. 

Although  the  variability  over  all  words  is  also  high,  the 
average  percentage  of  correct  responses  for  the  overt  condition  is  re- 
duced severely  for  all  Ss  relative  to  the  best  word,  with  only  four  of 
the  seven  Ss  achieving  a statistically  significant  percentage  of  correct 
responses.  Thus,  the  overt  condition  is  generally  superior  to  the  covert 
condition  for  both  the  best  word  and  over  all  words.  Nevertheless,  a 
sufficient  number  of  covert  responses  were  significantly  correctly  classi- 
fied to  indicate  the  feasibility  of  on-line  computer  classification  of 
covert  speech  or  verbal  thinking. 


Parameter  Results 


Using  the  data  base  given  in  Table  12  as  a reference,  we  now 
describe  the  variations  on  these  results  due  to  the  following  parameters 
studied : 


• Subjects 

- Correct  classification  rates  were  irvariably  greater 
in  trained  subjects  than  naive  suljects. 

- Practice  was  a vital  element  in  improving  classifica- 
tion, particularly  for  covert  EEG  responses,  in  trained 
or  naive  subjects. 

• Stimulus  presentation 

- EEGs  associated  with  repetitively  presented  words  were 
significantly  more  often  correctly  classified  than 
randomly  presented  words,  suggesting  that  anticipation 
or  "set"  improves  the  consistency  of  cerebral  organiza- 
tion of  the  EEG  associated  with  a particular  woi  d. 

In  one  set  of  experiments,  a series  of  words  were  used 
with  which  visual  images  were  associated.  For  example, 
overt  and  covert  responses  to  the  word  "airplane"  were 
classified  both  upon  presentation  of  the  word  "airplane" 
and  upon  presentation  of  a picture  (slide  shown  for  1 sec) 
of  an  airplane.  In  almost  all  S^s,  the  EEG  response  was 
correctly  classified  more  often  when  the  picture  was 
used  than  the  word,  by  10  to  1570. 
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Responses 

- EEGs  associated  with  overt  speech  had  higher  percent- 
ages of  correct  classification  than  those  associated 
with  covert  speech.  This  result,  in  large  part,  is 
due  to  the  difficulty  in  reproducing  a word  silently 
in  the  same  way  and  at  the  same  time  relative  to  the 
presentation  of  the  stimulus  without  the  audio  feed- 
back present  when  reading  a word  aloud,  since  highly 
trained  Ss  did  as  well  with  covert  words  as  overt 
words. 

- Single  "best  words"  usually  were  classified  correctly 
a high  percentage  of  the  time  (.70  to  100%),  indicating 
the  strong  feasibility  of  improving  overall  correct 
classification  rates.  This  may  be  contrasted  with  the 
average  correct  classification  rate  across  all  words 
of  20  to  297o  (though  one  of  our  Ss  typically  performed 
in  the  high  70s  and  80s). 

Record ing 

- Multiple  EEG  recording  s^tes  are  superior  as  classifier 
locations,  when  taken  together  either  as  averaged  re- 
sponses or  strung  "in-line"  as  a single  response,  than 
any  single  electrode.  This  is  due,  no  doubt,  to  the 
variability  within  and  between  Sis  at  a given  recording 
site. 

- No  single  electrode  site  appears  to  be  superior  to  any 
other;  however,  over  all  subjects  and  sessions,  temporal 
sites  (T3,  T4,  or  T5,  T6)  are  superior  to  frontal  or 
central  sites.  Occipital  and  parietal  areas  give  no 
better  than  chance  correct  classifications. 

- Generally,  averaged  le f t hemisphere  EEGs  were  classified 
correctly  more  often  than  averaged  right  hemisphere  EEGs, 
and  more  than  the  EEG  averaged  over  both  hemispheres. 

- A group  of  EEG  electrode  responses  can  be  classified 
as  one  response  by  either  averaging  all  individual 
responses  together  or  by  leaving  the  individual  responses 
alone  but  stringing  them  together  as  a single  response. 

The  latter  method  is  far  superior  to  the  averaged  method 
in  that,  for  both  overt  and  covert  responses,  the  correct 
classification  rate  for  the  averaged  method  is  about  20 

to  29%,  whereas  for  in- ] i ne  responses  the  rate  is  typically 
above  50%,  correct  responses,  even  across  all  words. 


Data  manipulation 
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- Signal  smoothing  (a  low-pass  filter)  provides  signifi- 
cantly higher  classifications  than  no  smoothing,  indi- 
cating that  the  information  for  classification  is  in 
the  low-frequency  range  of  the  EEG. 

- Exponential  decay  updating  of  templates  with  new  input 
signals  is  superior  to  no  updating  or  updating  by 
simple  averaging. 

- A short  EEG  epoch  of  about  500  msec  provides  a higher 
rate  of  correct  classifications  than  do  other  epochs 
collected  before  or  later  than  200  msec  after  the 
stimulus  presentation. 

- Mean-square  or  root-mean-square  classification  of  EEG 
responses  is  slightly  superior  to  correlational 
analysis . 

- Machine  time  justification  of  responses,  to  correct 
for  temporal  variation,  provides  fewer  correct  classi- 
fications than  no  time  justification  at  all  and 
signi ficantly  fewer  than  if  individual  signals  are 
time  justified  by  visual  means.  For  example,  the 

data  in  Figure  19  are  overlays  of  separate  EEG  responses 
at  each  electrode  site  for  each  of  the  seven  words. 

Each  response  has  been  time  justified  to  all  other 
responses  of  a group  by  visual  pattern  recognition. 

The  percentages  of  correct  responses  for  this  group 
of  data  increased  from  an  average  of  297o  for  machine 
time  justification  to  an  average  of  61°/„  after  visual 
time  justification. 


Concluding  Statement  About  On-Line  Analysis 

On-line,  real-time  computer  analysis  of  the  EEG  coincident  with 
overt  and  covert  speech  is  entirely  feasible,  although  it  is  not  yet 
sufficiently  reliable  to  be  used  in  a practical  system.  Examination  of 
Figure  19  shows  that  each  EEG  signal  pattern  is  uniaue  relative  to  indi 
vidual  words,  and  most  of  the  relevant  parameters  for  optimum  computer 
classification  are  evident  in  the  figure: 

(1)  EEG  epochs  must  be  as  short  as  the  verbal  response  itself 
and  must  be  collected  during  the  period  of  the  verbal 
response. 
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FIGURE  19  VISUAL  PATTERN  RECOGNITION  AND  TIME  JUSTIFICATION  FOR  SUBJECT  SM,  SESSION  1.  CLASS  6 


(2)  More  than  one  electrode  should  be  included  in  the  analysis, 
and  all  EEG  signals  analyzed  "in  line"  as  though  they  were 
one  response. 

(3)  Time  justification  by  machine  must  use  similar  pattern  recog- 
nition techniques  as  the  human  eye. 

(^0  Signals  should  be  smoothed  to  eliminate  higher  frequency 
"nois<  ." 

(5)  Templates  should  be  updated  with  new,  incoming  signals, 
using  an  exponential  decay  method. 


(6)  A time-series  analysis  using  a mean-square  or  root-mean- 
square  difference  algorithm  is  preferred  over  either  a 
correlational  technique  or  a frequency-domain  analysis. 

(7)  Subjects  should  be  trained,  should  be  practiced,  and 
should  have  a "set,"  or  anticipation  of  what  they  are 
to  say  or  verbally  think. 

(8)  For  some  forms  of  biocybernetic  communication,  pictures 
or  visual  images  might  be  used  to  good  advantage  along 
with  the  uttered  or  thought  word. 
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DISCUSSION  OF  BIOCYBERNETIC  COMMUNICATION 

The  purpose  of  this  project  was  to  test  the  feasibility  of  design- 
ing a biocyberne tic  communication  system,  whereby  the  human  brain  is 
closely  coupled  to  a computer  for  direct,  real-time,  man-machine  inter- 
action. We  conclude  from  our  results  that  it  is  feasible  to  use  the 
human  EEG  coincident  with  overt  and  covert  speech  as  inputs  to  a com- 
puter for  such  communication.  However,  we  also  conclude  that,  without 
additional  research,  the  EEG  is  not  adequate  for  the  design  of  a prac- 
tical operating  system;  indeed,  other  methods  than  those  employed  here 
may  prove  superior. 

Nevertheless,  enough  information  has  been  obtained  during  this  proj- 
ect to  specify  the  optimum  parameters  to  use  for  an  EEG-operating  system 
and  to  suggest  future  research  toward  that  end.  Our  results  show  con- 
clusively that  consistent,  repeatable  patterns  exist  in  the  EEG  during 
overt  speech  (for  example,  see  Figure  19)  and  covert  speech  (Figures  13 
and  14)  and  that  a computer  can  recognize  these  patterns  a statistically 
significant  percentage  of  the  time. 

The  major  problems  to  be  solved  in  using  the  approach  used  for  this 
project  are  those  of  variability.  For  example,  some  subjects  consistently 
have  high  percentages  of  correct  computer  classification  of  their  EEG 
during  both  overt  and  covert  speech,  as  high  as  857,  over  all  words  overtly 
spoken  and  as  high  as  72%  over  all  words  covertly  spoken.  In  these  same 
individuals,  EEGs  associated  with  individual  words  frequently  are  classi- 
fied correctly  100%  of  the  time,  regardless  of  the  number  of  times  re- 
peated (as  many  as  25  times  in  one  S)  . In  these  individuals,  the  pat- 
terns of  EEG  activity  are  consistent  over  time--in  fact  as  much  as 
18  months--so  that  EEGs  obtained  today  may  be  calssified  correctly 
against  templates  obtained  a year-and-a-half  ago. 

As  described  in  Part  II,  these  individuals  also  appear  to  have  a 
strong  hemispheric  lateralization  for  speech,  suggesting  that  they  pos- 
sess a more  consistent  pattern  of  cerebral  organization  than  others  who 
do  not  have  strong  hemispheric  lateralization  for  speech  (for  example, 
stutterers)  and  whose  EEGs  frequently  are  classified  correctly  by  the 
computer  at  no  better  than  chance  expectancy.  The  latter  group  of  in- 
dividuals represent  the  other  extreme  of  our  sub jects--those  showing 
pronounced  variability  in  the  EEG  in  time  and  amplitude.  Occasionally 
the  EEGs  of  these  individuals  have  consistent  patterns;  and,  during  these 
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periods,  computer  classification  improves,  as  when  actual  pictures  of 
objects  are  used  as  stimuli  rather  than  a word  representing  the  object. 
However,  these  patterns  do  not  persist,  so  that  during  a given  experi- 
mental session  there  may  be  a high  percentage  of  correct  classifications 
for  a time,  followed  by  a decline,  suggesting  that  the  pattern  of  cere- 
bral organization  has  shifted  or  drifted.  Some  evidence  exists  in  our 
data  to  suggest  that,  for  these  individuals,  a repeatable  pattern  will 
occur  when  the  stimulus  word  has  some  significance  to  the 

In  between  these  two  extremes  are  other  individuals  who  show  fairly 
consistent  EEG  patterns  that  are  classified  correctly  by  the  computer 
on  some  words  but  not  on  others,  on  some  days  but  not  on  others.  When 
the  data  of  all  three  groups  are  combined,  it  is  not  surprising  that  the 
overall  classification  rates  are  not  as  high  as  desired--on  the  average, 
about  357.  correct  classifications  on  overt  data  and  277,  on  covert  data. 
Both  sets  of  data  are  statistically  significant  (at  the  0.01  and  0.05 
levels,  respectively),  but  they  are  hardly  significant  enough  to  serve 
as  a basis  for  design  of  a practical  operating  system. 

It  would  be  a serious  error,  however,  to  discount  the  possibilities 
of  a biocybernetic  communication  system  based  on  these  combined  scores, 
since  there  are  more  individuals  with  scores  in  the  70  to  907,  correct 
classification  rates  than  in  the  10  to  157.  range.  Rather,  we  need  to 
determine  why  some  individuals  have  more  consistent  patterns,  why  some 
do  not,  and  how  consistent  patterns  might  be  obtained  or  even  induced 
in  those  who  do  not.  The  studies  on  hemispheric  laterality  described 
in  Part  II  point  to  ways  for  resolving  this  problem. 

Given  consistent  EFG  patterns,  a persistent,  almost  constant  error 
remains  in  machine  classification  of  the  EEG.  Much  of  this  might  be  due 
to  the  methods  we  used  in  pattern  recognition  or  to  the  amount  of  data 
provided  to  the  computer.  For  example,  we  know  that  if  six  to  eight 
electrodes  are  employed,  and  their  EEG  responses  are  strung  together  in 
line  as  a single  response  six  to  eight  times  as  long  as  a single  response, 
significantly  higher  correct  classification  rates  are  obtained  than  when 
either  a sing’e  electrode  is  used  or  when  all  electrode  responses  are 
averaged  together  a single  response  of  unit  length.  This  "in-line" 
procedure  essentially  increases  the  number  of  variables  employed  in  the 
classification,  giving  equal  weight  to  all  variables,  thus  suggesting 
that  a multivariate  approach  may  be  more  successful  than  the  approach 
used  here.  Also,  we  have  improved  correct  computer  classification  rates 
significantly  by  using  human  visual  pattern  recognition  to  time-justify 
signals.  Thus,  if  machine  time  justification  could  mimic  the  human  pat- 
tern recognition  algorithm,  much  of  our  present  variability  would  be 
reduced . 
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Another  major  source  of  variability,  especially  in  the  covert  re- 
sponses, is  the  naivete  and  training  of  the  Ss  and  their  "set"  to  respond. 
That  is,  sophisticated  _Ss  with  considerable  training  have  consistently 
higher  correct  computer  classifications  scores  than  naive,  untrained 
subjects.  Furthermore,  when  the  Ss  know  in  advance  what  word  they  are 
to  speak  or  think,  correct  classification  rates  are  higher.  This  cortical 
organization  for  saying  or  thinking  a word,  as  reflected  in  the  Ss'  EEG 
patterns,  is  more  consistently  evoked  when  sufficient  time  exists  for 
that  organization  to  occur,  and  the  Ss ' "set"  to  respond  is  well  formed. 


These  observations  indicate  that  the  optimum  parameters  required  for 
a consistently  accurate  computer-brain  intercommunication  system  are  as 
follows : 


(1) 


(4) 


(5) 


(6) 


(7) 


(8) 


(9) 


Ss  who  have  strong  hemispheric  lateralization  for  language 
should  be  chosen.  This  may  be  predetermined  by  dichotic 
listening  tests  (see  Part  II). 


(2) 


As  many  as  six  to  eight  electrodes  should  be  used  on  the 
side  of  cerebral  dominance  for  language,  with  emphasis  on 
cortical  areas  involved  in  speech-,  word-,  and  auditory- 
association  areas. 


(3) 


Ss  should  be  trained  in  the  operating  system,  especially 
temporal  training  for  covert  responses,  to  speak  or  think 
words  as  consistently  as  possible. 


Ss  should  know  what  word  will  be  said  or  thought  next;  i.e, 
they  should  have  a set  to  respond. 


Words  for  speaking  or  thinking  should  have  some  meaning 
for  the  J>,  or  should  be  used  in  a meaningful  context. 


Data  collected  from  each  EEG  electrode  should  be  limited 
as  nearly  as  possible  to  the  period  of  time  required  for 
actual  utterance  or  thought. 


Data  from  all  EEGs  should  be  strung  together  in-line,  as 
a single  response,  and  should  be  smoothed  sufficiently 
to  eliminate  high-frequency  components. 

Pattern  recognition  algorithms  the  computer  employs  should 
time-justify  the  single,  combined  response,  much  as  a 
human  observer  would  do  with  visual  pattern  recognition. 


Stored  patterns  should  be  "refreshed"  from  time  to  time 
(i.e.,  new  templates  should  be  formed  and  updated)  to 
take  account  of  "drifting"  cortical  organization. 
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Introduction 

As  indicated  in  the  General  Introduction,  we  have  no  a priori  evi- 
dence that  cortical  organization  for  language  production  or  verbal  think- 
ing resides  only  in  the  sites  we  have  chosen  to  place  our  EEG  electrodes. 
Indeed,  evidence  of  other  investigators  shows  that,  in  some  people,  cor- 
tical organization  for  speech  may  occur  in  locations  different  from  those 
we  have  investigated.  Therefore,  assuming  that  the  best  electrode  place- 
ment for  EEG  word-classification  will  be  located  over  that  cortical  area 
more  actively  engaged  in  producing  the  word,  it  is  important  to  be  able 
to  predict,  for  a given  subject,  in  which  hemisphere  verbal  functions 
are  most  likely  processed.  Also,  if  different  hemispheres  process  lan- 
guage function  in  different  individuals,  does  the  processing  itself  have 
a different  form?  Finally,  in  those  individuals  in  whom  speech  is  pro- 
cessed bilaterally,  may  we  expect  to  find  similar  measures  for  biocyber- 
netic  communication  as  in  individuals  in  whom  speech  is  processed  uni- 
laterally? 


Traditionally,  speech  localization  (i.e.,  hemispheric  asymmetry)  has 
been  assigned  to  that  hemisphere  opposite  to  the  handedness  of  the  indi- 
vidual involved.  Since  there  are  more  right-handed  people  than  left,  it 
has  been  assumed  that  the  left  hemisphere  is  predominantly  used  for  speech 
production.  This  concept  has  been  based  primarily  on  clinical  evidence 
from  neurosurgically  lesioned,  brain-damaged,  or  electrically  stimulated 
patients.  That  functional  brain  asymmetries  exist  has  been  well  docu- 
mented in  animals  (Rosensweig,  1951;  Hall  and  Goldstein,  1968),  and 
clinically  in  split-brain  human  patients  (see  Bogen  1969a  and  b,  for  an 
extended  review  and  bibliography),  whereas  anatomical  and  pathological 
verification  of  brain  asymmetries  has  been  demonstrated  in  over  200  hu- 
mans (Geschwind  and  Levitsky,  1968;  Glonig  et  al.,  1969).  In  additior, 
the  perception  of  language  also  appears  to  be  lateralized,  as  revealed 
by  dichotic  listening  tests  (Kimura,  1961,  1967) . "Dichotic  listening" 
refers  to  the  simultaneous,  bilateral  presentation  of  two  different 
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correlation  between  dichotic  listening  and  correct  computer  classification 
of  the  EEG  during  overt  speech,  used  data  previously  reported  in  these 
pages.  The  last  two  studies  on  EEG  asymmetry  and  word  classification  and 
target  recognition  and  EEG  alpha  asymmetry,  were  extensive  studies  in 
their  own  right. 


Dichotic  Correlations 


— s for  correlation  of  percentage  of  correct  classification  of  EEG 
response  words  with  dichotic  listening  scores  were  the  three  used  in  the 
off-line  experiments  (B,  C,  and  D)  and  the  five  used  for  the  Articulatory, 
Semantic,  and  Contextual  study  (U,  V,  W,  Y and  Z) . 


Each  subject  was  given  the  dichotic  listening  test  twice.  Percent- 
ages of  correct  responses  on  the  dichotic  tests  were  obtained  by  dividing 
the  number  of  correct  left  and  right  ear  responses  by  the  total  number 
of  stimuli.  Arbitrarily,  for  each  of  the  two  dichotic  tests,  left  ear 
scores  were  subtracted  from  right  ear  scores,  and  the  two  differences 
were  averaged  for  each  subject.  Thus,  a positive  average  difference  for 
a subject  indicated  right  ear  dominance,  whereas  a negative  average  dif- 
ference for  a subject  indicated  a left  ear  dominance.  According  to  past 
research  with  dichotic  listening  (see  discussion  avove),  right  ear  dom- 
inance signifies  left  cerebral  dominance  for  language  and  vice  versa. 


The  percentage  of  correct  classifications  of  EEG  responses  for  the 
right  and  left  hemispheres  (electrodes  T6  and  T5  respectively)  for  all 
Ss  on  the  two  language  recording  sessions  were  treated  similarly.  That 
is,  computer  classifications  for  EEG  responses  during  language  tasks 
were  obtained  for  each  of  the  eight  Ss  on  two  separate  sessions.  The 
percentages  of  correct  responses  for  the  right  hemisphere  electrode  (T6) 
were  subtracted  from  the  percentage  of  correct  responses  for  the  left 
hemisphere  electrode  (T5)  for  each  session,  and  the  differences  were 
averaged  for  each  S.  Thus,  a positive  average  difference  for  a subject 
indicated  a left  hemisphere  dominance  for  correct  classification  of  EFG 
responses  during  the  language  task  and  vice  versa. 


The  results  are  given  in  Table  13.  Ss  (first  column)  were  ranked 
from  the  maximum  left  hemisphere  EEG  dominance  to  the  maximum  right 
hemisphere  EEG  dominance.  A score  of  zero  indicates  equality.  Average 
difference  scores  for  EEG  dominance  for  each  £ are  given  in  the  second 
column,  and  the  corresponding  average  difference  scores  from  the  dichotic 
listening  tests  for  ear  dominance  are  given  in  the  third  column.  As  in- 
dicated, three  of  the  eight  Ss  were  left  EEG  hemisphere  dominant  (and 
right  eared),  or  was  bilateral  (and  equal  eared),  and  four  were  right 
EEG  hemisphere  dominant  (left  eared). 
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Table  13 

RANK  ORDER  COMPARISON 
BETWEEN  HEMISPHERIC  EEG  DOMINANCE 
AND  EAR  DOMINANCE* 


Sub  ject 


Rank  Order  Correlation  = 
0.786,  0.01  < p < 0.05 
Average  Difference 


EEG  Dominance 


Ear  Dominance 


1 


c 

V 
D 
U 

Y 
W 
B 
Z 


+15.0 

+11.5 

+9.5 

+0.0 

-2.0 

-11.0 

-12.0 

-15.0 


+11.5 

+7.5 

+5.0 

+0.0 

-11.5 

-1.5 

-11.0 

-1.5 


See  text  for  definitions. 


;• 

■ 


L 


4 
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A rank  order  correlation  was  calculated  between  EEG  and  ear  domi- 
nance, with  a significant  Rho  of  0.786  (0.01  < p < 0.05).  Thus,  if  the 
dichotic  tests  actually  measure  lateralit)  of  language  function,  low 
computer  classification  scores  on  a given  electrode  may  simply  mean  that 
language  is  not  processed  under  that  electrode.  Conversely,  these  re- 
sults confirm  that  a dichotic  test  is  the  simplest  way  of  determining 
placement  of  EEG  electrodes  for  biocybernetic  communication.  These  re- 
sults are  also  significant  in  view  of  the  fact  that  all  eight  Ss  were 
chosen  because  they  were  right-handed  and,  therefore,  presumably  left 
hemisphere  dominant  for  speech.  Had  electrodes  been  used  more  extensively 
over  the  right  hemisphere  of  the  Ss  with  left  ear  dominance,  the  percent- 
age of  correct  EEG  response  classifications  probably  would  have  been 
higher  than  those  obtained. 
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EEG  Asymmetry  and  Word  Classif ication* 

Introduction 

About  80  to  907o  of  right-handed  individuals  process  language  in 
their  left  cerebral  hemispheres  and  music  and  spatial  tasks  in  their 
right  hemispheres  (Wada,  1949;  3ogen  and  Bogen,  1969).  The  remaining 
10  to  20%  of  dextrals  and  40  to  50%  of  sinistrals  seem  to  have  the  re- 
verse of  this  pattern  of  organization  or  have  these  functions  more  bi- 
laterally represented  in  both  hemispheres  (Branch  et  al.,  1964;  Goodglass, 
1972;  Glonig  et  al.,  1969).  Verbal  and  nonverbal  functions  may  have  be- 
come represented  in  opposite  hemispheres  through  evolution  to  reduce  the 
interference  of  one  cognitive  system  with  the  other  (Levy,  1969) . Levy 
and  others  have  suggested  that  variations  in  the  laterality  patterns  of 
these  processes,  such  as  bilaterality,  may  result  in  inefficie.it  pro- 
cessing (Levy,  1969;  Landsell,  1969;  Miller,  1971;  Galin,  1973).  One  way 
to  test  this  hypothesis  is  to  compare  the  pattern  of  laterality  in  normal 
right-handed  Ss  with  the  patters  in  Ss  who  have  a known  inefficiency  of 
function,  such  as  stuttering,  that  may  be  related  to  inadequately  estab- 
lished hemispheric  specialization  (Orton,  1927;  Travis,  1931;  Bryngelson 
and  Rutherford,  1937;  Jones,  1966;  Van  Riper,  1971). 

Although  stuttering  does  diminish  with  some  psychological  therapies, 
to  view  it  in  purely  psychological  terms  is  to  disregard  much  evidence  of 
organic  etiology.  For  example,  there  are  many  indications  of  a heredi- 
tary factor.  Stuttering  is  known  to  run  in  families  (especially  those 
with  left-handed  members)  and  is  four  times  as  prevalent  among  males  as 
females.  Also,  it  is  insidious  in  onset,  occurring  between  the  ages  of 
two  and  five  when  a child  is  first  learning  language  and  is  rarely  re- 
ported as  originating  at  any  other  age  unless  as  a result  of  frank  brain 
injury. 

There  are  essentially  two  schools  of  thought:  one  considers  stut- 

tering a psychological  problem  and  one  regards  it  as  an  organically  based 
disability.  In  the  1930s,  a great  deal  of  attention  was  directed  toward 
a theory  advanced  by  Orton  (1927)  and  Travis  (1931)  that  stuttering  was 
caused  by  a lack  of  cerebral  dominance,  resulting  in  a rivalry  between 
the  two  hemispheres  for  the  control  of  the  bilateral  musculature  involved 
in  speech.  Evidence  supporting  this  theory  came  from  many  sources.  A 
change  of  handedness  sometimes  is  accompanied  by  the  onset  of  stuttering 
in  very  young  children,  whereas  reverting  to  the  original  hand  has  been 


* 

This  study  was  carried  out  as  part  of  a Ph.D.  thesis  by  Dr.  Jeanmne 
Herron  for  the  Department  of  Anatomy,  Tulane  University,  New  Orleans,  LA. 
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known  to  alleviate  it.  Tests  of  peripheral  preferences  (eye,  hand,  foot) 
requently  have  indicated  some  lack  of  hemispheric  lateralization  in 
' utterers‘  Bryngelson  and  Rutherford  (1937)  reported  that,  in  a normal 

trous  However"  ^ ^ ^ Uft  handed'  and  8%  arabidex‘ 

’ lil  a stuttering  population,  the  distribution  was  617 

right  handed,  47c  left  handed,  and  347,  ambidextrous. 

stuttered"3  ^ iBdiC*te  S°me  degree  of  incomplete  lateralization  in 
stutterers  are  numerous  and  conflicting.  To  resolve  this  problem  it  is 

function"  ? ^ ^ >afel*  aBd  ^ the  Latioi  of 

unctions  known  to  be  asymmetrically  organized  in  the  cortex. 

Recent  studies  have  shown  that  the  EEG  can  be  used  for  this  purpose 
ower  in  the  alpha  frequency  band  of  the  EEG  (8  to  12  Hz)  attenuates 
uring  cognitive  tasks  such  as  mental  arithmetic  vGlass,  1964,  1970- 
erger  1929;  Adrian  and  Matthews,  1934).  Other  investigators  have 
demonstrated  that  variations  in  alpha  activity  during  hemisphere-related 

— 9e?n  " an3lyZ:  aSymmetry  °f  “ tical  functioa  (Galin  and 
Ornstein,  1972;  Morgan  et  al.,  1971;  McKee  et  al.,  1973). 

Another  technique  for  assessing  language  and  music  laterality  is  the 
multaneous  presentation  o,  competing  verbal  or  musical  signals  to  both 
ears  (dichotic  listening) . Normal  right-handed  Ss  prefer  their  right  ear 

(LurrCW6«“Uli-<kr°‘db*ne’  19M)  and  thelr  left  ear  f“  stimuli 

1“  ’ 9“);  Rleht-ear  preference  for  language  la  interpreted  as  re- 

ecting  verbal  processing  In  the  left  hemisphere.  More  stutterers  have 
been  found  to  have  a left  ear  preference  for  verbal  material  than  contrL 

this  1Sten1"8  teS"  (Curr>’’  196?;  Perrin.  1969;  Quinn,  1972),  but 

n ing  was  not  confirmed  by  Sussman  and  MacNeilage  (1974). 

This  experiment  was  designed  to  compare  the  lateralization  of  lan- 
guage and  music  abilities  in  the  cortical  hemispheres  of  stutterers  and 

cnalTofrStheanEdEG°dUSe  ^ computer  classifi- 

cation of  the  EEG  during  speech.  Hemispheric  asymmetry  for  these  fus- 
ions was  measured  by  analyses  of  EEG  activity  in  the  8-  to  12-Hz 
requency  band  from  bilaterally  placed  homologous  leads,  and  these  mea- 
sures were  correlated  with  behavioral  scores  of  the  same  subjects  during 

ob^ined"r  Vi  ^°tLC  llStening  task‘  EEG  classification  then  was 
7 Ti!!8  off-line>  root-mean-square  method  described  in  the 

econd  and  Third  Semi-Annual  Technical  Progress  Reports. 
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Methods 

Sub  jects 

Ss  were  six  adult,  right-handed  stutterers  and  eight  adult, 
right-handed  nonstutterers,  all  of  whom  were  rale,  except  the  eighth 
nonstutterer,  BA.  Ages  ranged  from  18  to  37  years.  Histories  of  each 
£5  were  taken  which  included  a handed  less  questionnaire  and  information 
about  incidence  of  familial  sinistrality  and  S/s  preferred  foot,  ear, 
and  sighting  eye.  No  S was  admitted  to  the  study  who  had  any  history  of 
severe  anoxia  or  febrile  disease  or  brain  injury. 


EEG  Experiment 

Electrodes  and  Electrode  Placements--Scalp.  reference,  and 
ground  electrodes  were  Beckman  silv.  . , silver-chloride  standard  disk 
skin  electrodes.  Locations  T3,  T4,  T5,  and  T6  of  the  10/20  system  of 
EEG  recording  (Penfield  and  Jasper,  1954)  were  used  with  the  reference 
electrode  at  the  vertex  at  Cz,  and  the  ground  was  over  the  right  mastoid 
process . 

EEG  signals  were  recorded  on  a Beckman  Type  R Dynograph. 
Amplitude  was  normalized  by  setting  all  channels  to  the  same  gain.  The 
time  constant  throughout  all  sessions  was  0.1  sec.  These  data  also  were 
recorded  on  an  Ampex  SP-300  analog  tape  recorder  and  then  sent  through 
the  data  analysis  system  using  a Digital  Equipment  Corporation  Linc-8 
laboratory  instrument  computer  with  analog  to  digital  converters,  a 
Pertec  digital  tape  recorder,  and  a CDC-6400  computer  (see  data  analysis 
section  below) . Calibration  procedures  to  ensure  equality  of  each  chan- 
nel were  run  before  and  after  each  session. 


Stimulus  Ma ter i a 1 s - -Cogni t ive  tasks  designed  to  activate  cor- 
tical areas  in  left  and  right  hemispheres  were  presetted  on  15  slides 
as  follows: 

(1)  Five  verbal  tasks--HIT,  COOL,  TIP,  HAD,  PUT. 

(2)  Five  music  tasks--illustrated  in  Figure  20,  pro- 
nounced with  neutral  "uh." 

(3)  Five  combined  music  and  verbal  tasks--same  music 
tasks  as  above  in  (2)  pronounced  with  the  words  in 
(1)  instead  of  with  the  neutral  sound  "uh,"  illus- 
trated in  Figure  20. 
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Stimulus  Presentation- -initiation  and  maintenance  of  a 5-sec 
trial  interval,  as  well  as  on-line  storage  of  each  trial  interval,  were 
provided  by  a Linc-8  program  (CNTLSTIM) . Figure  21  illustrates  an  actual 
chart  page  with  the  CNTLSTIM  inter  'al  on  Channel  7 and  the  voice  response 
on  Channel  8. 

Slides  were  changed  manually  during  the  intervals  between  each 
of  the  15  stimulus  presentations.  A single  random  presentation  of  each 
of  the  15  stimuli  constituted  one  set  of  trials.  Following  the  presenta- 
tion of  one  complete  set,  the  15  slides  were  randomized  and  made  ready 
for  the  next  set.  Further  presentations  of  the  same  15  stimuli  were 
conducted  in  an  identical  manner  for  a ti  al  of  ten  trial  sets.  Thus, 
for  each  S,  the  data  were  based  on  EEG  sigi.als  from  four  electrodes  for 
two  recording  sessions,  where  each  session  consisted  of  ten  repetitions 
of  each  of  five  words,  five  musical  phrases,  and  five  combined  tasks. 

This  resulted  in  a total  of  1,200  (4  X 2 X 10  X 15)  electrophysiological 
responses  of  5 sec/S,  or  16,800  for  the  entire  experiment.  The  duration 
of  an  average  session  was  1-1/2  hours.  Intervals  between  sessions  were 
typically  two  weeks  (range  of  one  week  to  one  month) . 


Data  Collection  and  Analysis--A  data  block  comprised  the  EEG 
data  collected  at  51  samples  per  second  during  the  5-sec  trial  interval. 

Only  data  points  91  through  218  (see  Figure  21--area  outlined 
by  dotted  lines),  which  included  the  major  part  of  the  actual  response 
period,  were  selected  from  the  data  blocks  on  the  Pertec  tapes  for  spec- 
tral analysis  to  reduce  variability  caused  by  the  nonresponse  portion  of 
the  EEG. 

A Fourier  power  spectral  analysis  to  determine  alpha  amplitudes 
was  run  off-line  on  the  CDC-6400  computer,  which  first  separated  the  fre- 
quencies (0  to  20  Hz)  into  33  "bins."  The  power  in  Bins  10  through  15, 
representing  the  alpha  frequencies,  was  summed  to  total  the  alpha  activity 
in  one  response  period  at  one  electrode  site.  The  totals  for  the  ten  ut- 
terances of  a given  response  were  then  averaged.  All  50  verbal  responses 
were  averaged,  all  50  music  responses  were  averaged,  and  all  50  combined 
responses  were  averaged.  Thus,  there  were  three  scores--music,  verbal 
and  combined  "average  alpha  amplitude"  (AAA)  computed  for  each  electrode 
site.  Verbal  to  music  (v'/M)  ratios  also  were  computed  for  each  electrode 
site  using  the  verbal/music  AAAs . 


Artifact  Contamination--The  computation  of  AAAs  was  performed 
without  any  editing  of  artifact.  It  was  assumed  that; 
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(1)  High-frequency  contamination,  s.nw  high-amplitude 
muscle  artifacts,  and  eye-movement  potentials  largely 
would  be  eliminated  by  the  fact  that  only  alpha  fre- 
quencies were  analyzed  which  exclude  all  frequencies 
outside  7.7  to  12.0  cps. 

(2)  Except  for  a comparison  of  raw  AAA  values  between 
stutterers  and  nonstutterers,  all  values  analyzed 
were  relative,  not  absolute  (hemispheres  relative 
to  each  other,  or  tasks  relative  to  each  other). 
Artifact  contamination,  therefore,  would  have  to  con- 
sistently influence  only  one  of  the  paired  electrodes, 
on  only  one  particular  task  (over  a randomly  ordered 
total  of  50  verbal,  50  music,  or  50  combined  trials) 
to  affect  the  results. 


Dichotic  Listening  Experiment 

Sub jects--Ss  were  the  same  as  those  in  the  EEG  experiment.  All 
Ss  were  given  a Rudmose  Clinical  Audiometric  test  to  confirm  normal  and 
equal  hearing  in  both  ears.  No  S was  included  who  showed  significant 
hearing  loss  in  either  ear. 


Equipment--A  Teac  3300-11  stereo  tape  deck  (Teac  Corporation) 
and  Nova-10  stereo  headphones  (Realistic)  were  used.  The  output  at  the 
headphones  was  calibrated  at  70db  SPL  by  putting  the  earphones  on  a flat 
plane  coupler,  reading  the  VTVM,  and  adjusting  the  amplifier  to  the  ap- 
propriate settings.  The  audio  tape  (provided  by  Dr.  Peter  MacNeiloge 
and  made  by  Mr.  David  Hakes,  both  of  the  Department  of  Linguistics, 
University  of  Texas  at  Austin)  contained  six  stop  constant-vowel  syllables: 
/b, d, g, p, t, k/,  each  followed  by  a vowel  /ae/.  These  were  arranged  for 
60  dichotic  trials.  The  60  trials  were  given  twice;  the  second  time  the 
earphones  were  reversed.  Subjects  were  instructed  always  to  answer  with 
tjwo  different  consonants,  and  to  wri  te  the  one  they  were  most  sure  of 
first.  The  number  of  correct  responses  was  totalled  for  the  left  ear 
and  for  the  right  ear.  The  difference  between  the  total  right  correct 
and  the  total  left  correct  (R-L)  was  the  dichotic  score  for  each  subject. 
The  scores  from  the  two  sessions  were  averaged. 
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Results 


Average  Alpha  Amplitude 

The  mean  AAA  values  (in  arbitrary  units)  for  stutterers  and 
nonstutterers  during  all  three  tasks  are  shown  in  Figure  22.  To  evalu- 
ate the  reliability  of  the  data  between  sessions,  a t-test  was  done  com 
paring  the  two  sessions  at  e uch  electrode  and  for  each  task.  There  was 
no  significant  difference  between  the  two  sessions  in  either  group. 

The  AAAs  of  stutterers  are  significantly  higher  at  all  four  sites  and 
during  both  tasks  than  the  AAAs  of  the  nonstutterers  (p  < 0.02,  one- 
tailed  t-test) . 


Dichotic  Listening  Tests 


Although  the  difference  in  AAA  between  stutterers  and  nonstut- 
terers is  significant,  it  indicates  little  cr  nothing  about  the  lateral 
organization  of  language  in  either  group.  Therefore,  dichotic  listening 
scores  were  examined  next  to  determine  whether  the  groups  differed  in  ear 
preference . 

The  dichotic  listening  scores  were  computed  by  subtracting  the 
total  number  of  correctly  reported  CV  syllables  presented  to  the  left 
ear  from  the  total  number  of  correctly  reported  CV  syllables  presented 
to  the  right  ear  for  each  £.  These  scores  were  averaged  for  the  two 
sessions  and  are  listed  in  the  fourth  column  of  Table  14  with  other  data 
reported  below.  Right-ear  scores  were  significantly  higher  than  left- 
ear  scores  in  both  groups  (p  < 0.02,  Mann  Whitney  U test).  There  was  no 
significant  difference  between  the  right-minus-left  scores  of  the  two 
groups.  That  is,  the  R-L  differences  were  not  smaller  among  the  stutter- 
ers, as  has  been  reported  by  some  other  studies;  and  there  was  no  greater 
incidence  of  left-ear  preference  among  the  stutterers  than  among  the  non- 
stutterers . 


When  all  Ss  were  viewed  as  individuals  in  terms  of  their  ear 
preferences  on  the  dichotic  listening  teat,  four  distinct  new  groups 
emerged.  Of  the  eight  nonstutterers,  five  clearly  preferred  their  right 
ear.  These  five--LR,  BA,  BL,  NC,  and  MP— all  had  average  dichotic  listen- 
ing scores  higher  than  +3.  i.his  value  is  taken  ns  an  arbitrary  cutoff 
point  for  "decided  ear  preference."  Three  of  the  nonst atterers  were  not 
right-eared;  that  is,  they  had  average  scores  of  +3  or  lower.  Two  of 
them,  JD  and  PG,  did  not  show  a strong  preference  for  either  ear,  and 
the  third,  JH,  clearly  preferred  his  left  ear. 
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Table  14 


f 


VERBAL-TO 

-MUSIC  RATIOS 

(V/M)  OF 

AVFRAGE  ALPHA  AMPLITUDE 

V/M  T4 

V/M  T6 

Ss 

V/M  T3 

V/M  T4 

-V/M  T3 

V/M  T5 

V/M  T6 

-V/M  T5 

Right-Eared  Nonstuttevers 

LR 

0.74 

0.97 

0.23 

1.06 

1.14 

0.20 

BA 

0.79 

0.88 

0.09 

0.93 

1.03 

0.10 

BL 

0.85 

1.10 

0.25 

0.97 

1.21 

0.23 

NC 

0.87 

0.94 

0.07 

0.89 

0.88 

-0.01 

MP 

0.98 

1.07 

0.09 

0.69 

1.01 

0.32 

Mean 

0.85 

1.00 

0.15 

0.91 

1.06 

0.15 

Nonright 

-Eared  Nonstutterers 

JD 

1.12 

1.13 

0.01 

1.15 

1.32 

0.17 

PG 

1.15 

1.05 

-0.10 

1 .10 

1.09 

-0.01 

JH 

1.10 

0.98 

-0.12 

1.27 

1.20 

-0.07 

Mean 

1.12 

1.05 

-0.07 

1.18 

1.20 

0.03 

Right- 

Eared  Stutterers 

JN 

1.08 

1.03 

-0.05 

1.39 

1.48 

0.09 

RR 

1.10 

0.92 

-0.18 

1.01 

1.05 

0.04 

TD 

0.93 

1.15 

0.22 

1.23 

1.10 

-0.13 

BC 

1.11 

1.67 

0.56 

1.61 

2.42 

0.81 

Mean 

1.06 

1.20 

0.14 

1.31 

1.51 

0.20 

Nonright-Eared 

Stutterers 

DS 

1.10 

1.23 

0.13 

1.19 

1.31 

0.12 

TG 

0.94 

0.96 

0.02 

1.32 

1.18 

-0.14 

Mean 

1.03 

1.10 

0.08 

1.25 

1.24 

-0.01 

Among  the  six  stutterers,  four  strongly  preferred  their  right 
ear  (JN,  RR,  TD,  and  BC) , two  showed  nonright-ear  preference  (DS  and  TG) , 
and  none  showed  a decided  left-ear  preference.  Since  all  Ss  were  selected 
on  the  basis  of  right-handedness  to  achieve  homogeneity  of  laterality 
within  groups,  a further  selection  was  made  on  the  basi‘  of  ear  prefer- 
ence. Both  stutterers  and  nonstutterers  were  separated  i ito  two  subgroups, 
"right-eared"  and  "rionright-eared . " 


Verbai-to-Music  (V/M)  Ratios 
Within  and  Between  Groups 


V/M  ratios  were  calculated  to  determine  the  relationship  between 
the  two  tasks  at  a given  electrode.  This  approach  eliminates  the  vari- 
ability between  sites  that  may  occur  as  a result  of  skull  thickness  or 
other  factors,  and  each  site  serves  as  its  own  control.  For  example,  the 
difference  in  alpha  amplitude  that  occurs  at  the  left  Heschl's  gyrus  can 
be  seen  when  the  t*-sk  is  changed  from  verbal  to  music. 

V/M  ratios  at  all  four  electrode  sites  are  shown  in  Table  14. 

It  is  apparent  that  the  right-eared  nonstutterers  had  lower  scores  at 
T3  than  at  T4  or  T6  in  that  group  or  at  any  electrode  site  in  any  other 
group.  Statistical  comparisons  were  made  between  electrodes  T3  and  T4 
and  between  T5  and  T6  within  the  right-eared  nonstuttering  group  using 
t-tests.  T3  V/M  scores  were  significantly  lower  than  T4  V/M  scores 
(p  < 0.05).  Although  the  left  hemisphere  scores  were  significantly 
lower  than  those  from  the  right  in  this  group,  the  same  differences 
between  hemispheres  were  not  found  in  any  of  the  other  three  groups. 

Comparisons  were  then  made  at  each  electrode  site  between  the 
right-eared  nonstutterers  and  all  the  other  groups.  At  the  right  hemi- 
sphere leads,  T4  and  T6,  there  vias  no  difference  among  any  of  the  groups; 
however,  at  the  left  hemisphere  leads,  at  both  T3  and  T5,  there  were 
significant  differences  (p  < 0.(2,  Mann  Whitney  U test)  between  the  right- 
eared nonstutterers  and  each  of  the  other  three  groups.  (The  p value  for 
the  nonright-eared  stutterers  was  approaching  significance  for  T3,  but 
was  not  so  because  of  the  small  N.) 


Correlation  of  V/M  Scores  with  Dichotic 
Listening  Scores 

\ 

V/M  ratios  at  T3  might  be  expected  to  correlate  with  dichotic 
listening  scores  because  a high  R-L  score  on  the  dichotic  listening  test 
is  thought  to  indicate  verbal  processing  at  left  Heschl's  gyrus  (T3)  . 

The  lower  the  V/M  ratio  at  T3,  the  higher  the  dichotic  listening  score 
should  be;  and,  conversely,  the  higher  the  V/M  ratio  at  T3,  the  more  neg- 
ative the  dichotic  listening  score  should  be.  V/M  scores  at  T3  (ranked 
from  lowest  to  highest)  and  dichotic  listening  scores  (ranked  from 
positive  to  negative)  were  correlated  for  stutterers  and  nonstutterers 
using  the  Spearman  Rank  Correlation  Test.  These  scores  and  their 
ranks  for  the  two  measures  are  shown  in  Table  15.  A +0.81  correlation 
(p  = < 0.01)  was  found  between  V/M  ratios  at  T3  and  dichotic  listening 
for  nonstutterers,  but  a slightly  negative,  nonsignificant  correlation 
was  found  for  the  stutterers. 
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Table  15 


I 


RANK  CORRELATION  BETWEEN  T3  V/M  RATIOS 
AND  DICHOTIC  SCORES 


Dicho  tic 


ss 

V/M  T3 

Rank 

R-L 

Rank 

Nonstutterers 

LR 

0.74 

1 

10.5 

3 

BA 

0.79 

2 

17.0 

1 

BL 

0.85 

3 

11.0 

2 

NC 

0.87 

4 

4.5 

5 

MP 

0.98 

5 

10.0 

4 

JD 

1.12 

7 

-1.5 

7 

PG 

1.15 

8 

3.0 

6 

JH 

1 . 10 

6 

-10.5 

8 

Spearman  rg  = 0.81 

p = < 

0.01 

Stutterers 

JN 

1.09 

3 

14.5 

2 

RR 

1.11 

4.5 

10.5 

4 

TD 

0.93 

1 

12.5 

3 

BC 

1.12 

6 

18.0 

1 

DS 

1.11 

4.5 

-2.0 

6 

TG 

0.94 

2 

-1.5 

5 

Spearman  rg  = -0, 

,19  (NS) 

To  summarize,  right-eared  nonstutterers  as  a 

group  could  be 

distinguished  from 

stutterers 

as  a group 

on  the  basis  i 

of  two  different 

measures  of  their 

left  temporal  response 

to  the  verbal 

and  music  tasks : 

(1)  they  had  lower 

V/M  ratios 

at  T3  and  T5,  and  (2)  their  rank  correla- 

tion  between  T3  V/M  scores  and  dichotic  ear  preference 

was  positive  and 

significant,  whereas  no  such  correlation 

was  found  with  the  stutterers. 

However,  nonright- 

eared  nonstutterers  could  not  be  distinguished  from 

stutterers  on  the 

basis  of  any  results  shown  so  far. 

In  other  words,  no 

obvious  differences  existed  between  the  two  that  might 

explain  stuttering 

in  one  group  aid  not  in  the  other. 


I 


I 

\ 


I 


95 


Examination  of  data  produced  by  tne  "combined"  task  did  not 
yield  significant  results. 


Discussion 


EEG  Alpha  A symmetry 

It  is  of  considerable  interest  that  the  right-eared  nonstutter- 
ers were  the  only  group  to  produce  significant  differences  between  the 
V/M  ratios  of  the  two  hemispheres.  In  their  V/M  ratios,  stutterers  were 
significantly  like  nonright-eared  nonstutterers,  who  showed  no  signifi- 
cant differences  between  hemispheres,  and  unlike  right-eared  nonstutter- 
ers. However,  an  examination  of  Table  15  shows  that  the  reason  was  not 
because  the  differences  between  hemispheres  in  stutterers  and  nonright- 
eared nonstutter<»rs  were  consistently  small  (indicating  more  "bilateral" 
organization),  but  because  the  lowest  V/M  ratios  were  not  consistently 
in  the  same  hemisphere  for  ell  members  of  the  group,  as  they  were  in  the 
right-eared  nonstuttering  group. 


Correlation  Between  Dichotic  Ear  Preference 
and  EEG  Alpha  Asymmetry 

We  established  that  a significant  positive  correlation  exists 
between  dichotic  R-L  scores  and  the  V/M  ratios  at  T3  in  nonstutterers. 
Figure  23  shows  that  all  right-eared  nonstutterers  had  lower  T3  V/M 
ratios  than  T4  ratios  (as  would  be  expected  if  verbal  processing  takes 
place  at  T3) , and  in  Figure  24  that  two  of  the  nonright-eared  nonstut- 
terers had  higher  V/M  ratios  at  T3  than  at  T4  (as  would  be  expected  if 
verbal  processing  takes  place  at  T4) . 

EEG  alpha  during  hemisphere-related  tasks  previously  has  not 
been  correlated  with  dichotic  listening  scores  in  "normals"  or  in  any 
other  groups.  The  significance  of  the  rank  correlation  makes  this  one 
of  the  most  important  findings  of  this  research.  It  was  reasonable  to 
expect  that  a right-ear  preference  would  correlate  with  left  hemisphere 
verbal  processing  and,  conversely,  that  a left-ear  preference  would  cor- 
respond to  right  hemisphere  verbal  processing.  But  finding  a significant 
rank  correlation  was  surprising.  The  indication  from  these  results  is 
that  the  amount  of  alpha  asymmetry  correlates  with  the  extent  of  our  pref- 
erence. The  fact  that  these  measures  separated  "normal"  Ss  with  "typi- 
cal" lateral  organization  from  those  with  "reversed"  organization  make 
a powerful  diagnostic  tool--if  their  correlation  remains  constant  in 
future  studies,  and  if  they  can  predict  correctly  lateral  organization 
in  patients  tested  with  sodium  amytal. 
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FIGURE  24  V/M 


The  second  finding  shown  in  Table  15  is  that,  although  the 
nonstutterers  had  a significant  and  positive  correlation,  the  correla- 
tion in  the  stuttering  group  was  slightly  negative  and  nonsignificant. 

The  reason  that  no  correlation  existed  between  d^chotic  ear  preference 
and  V/M  T3  scores  in  stutterers  becomes  obvious  when  the  patterns  for 
individuals  are  examined  in  Figures  23  and  24.  To  compare  ear  prefer- 
ence and  V/M  scores  at  the  anterior  temporal  leads  in  individual  non- 
stutterers and  stutterers,  we  first  had  to  establish  which  individuals 
had  a definite  ear  preference.  All  right-eared  Ss  lu'd,  by  definition, 
an  ear  preference.  However,  the  nonright-eared  Ss  did  not,  except  for 
nonstutterer  JH  whose  R-L  score  was  -10  and  who  would  be  classified  as 
left-eared.  Therefore,  six  out  of  eight  nonstutterers  had  an  ear  pref- 
erence. Figure  23  shows  that,  in  each  case,  the  lowest  of  the  two  an- 
terior temporal  leads  was  contralateral  to  the  preferred  ear.  On  the 
other  hand,  four  stutterers  could  be  said  to  have  an  ear  preference.  Of 
these,  two  had  their  lowest  anterior  temporal  score  contralateral . and 
the  other  two  had  it  ipsilateral . to  their  preferred  ear.  (The  Fisher 
Test  of  Exact  Probabilities  can  be  used  to  determine  the  probability  of 
this  distribution;  in  this  case  it  was  0.05.) 

A seventh  stutterer  was  tested  only  once  because  he  could  not 
return  for  the  second  session.  For  this  reason,  and  because  he  performed 
several  tasks  with  his  left  hand,  he  was  not  included  in  the  analysis 
with  the  rest  of  the  stutterers,  although  his  data  support  the  general 
trend.  Both  left  hemisphere  V/M  ratios  were  higher  than  right  hemisphere 
ratios  (T3  = 1.27,  T4  = 1.06,  T5  = 1.27,  and  T6  = 1.24).  However,  this 
S was  quite  strongly  right-eared  on  the  dichotic  listening  test  (his  R-L 
score  was  +22),  an  indication  that  his  lowest  V/M  score  should  be  at  T3, 
not  T4.  If  he  were  counted  in  the  subject  population,  he  would  fall  into 
the  category  with  JN  and  RR  whose  lowest  anterior  temporal  scores  were 
ipsilateral  to  their  preferred  ear. 

It  is  puzzling  that  the  hemisphere  apparently  doing  the  most 
verbal  processing  was  ipsilateral  to  the  preferred  ear.  Either  these 
results  occurred  by  chance  or  mistake  or  the  two  conditions--( 1)  listen- 
ing to  a tape  through  earphones  and  (2)  listening  to  one's  own  voice 
feedback--are  comparable  in  nonstutterers  but  not  in  stutterers.  There 
is  evidence  that  the  latter  may  be  true  (Butler  and  Galloway,  1957;  Cherry 
and  Sayers,  1956;  Stromsta,  1958;  Tomatis,  1954,  1963;  Van  Riper,  1971). 


EEG  Asymmetry  and  Word  Classification  Experiment 

Data  obtained  during  the  preceding  experiments  were  analyzed 
by  computer  classification  of  the  EEG  during  speech  production  off-line 
on  the  CDC  6400,  using  the  root-mean-square  method  as  described  in  the 
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Third  Semi-Annual  Technical  Progress  Report  (September  1973).  Subjects 
then  were  grouped  according  to  their  dichotic  listening  tests,  as  de- 
scribed above,  and  group  comparisons  were  made  between  alpha  asymmetries 
and  percentage  of  correct  classification. 


Percentage  of  Correct  Classification  Scores  at  Known  Speech- 


Processing  Sites  Versus  Those  at  Other  Sites--Me  expected  that  the  per- 
centage of  classification  scores  would  be  higher  from  electrode  sites 
over  known  speech-processing  centers  than  from  other  cortical  sites.  In 
previous  experiments,  all  available  EEG  channels  had  been  used  to  collect 
data  from  these  centers.  In  this  experiment,  two  parietal  sites  also  were 
used  (P3  and  P4) , which  probably  were  active  in  the  processing  of  the 
visually  presented  stimulus  but  not  in  the  production  of  speech.  Table  16 
shows  the  percentage  of  classification  scores  from  temporal  and  parietal 
leads  in  four  groups  of  S s : (1)  right-eared  nonstutterers,  (2)nonright- 

eared  nonstutterers,  (3)  right-eared  stutterers,  and  (4)  nonright-eared 
stutterers.  The  parietal  sites  produced  mean  percentage  of  classifica- 
tion scores  at  chance  levels  only  (chance  = 207o) , whereas  the  means  for 
T3,  T4,  T5,  and  T6  of  the  right-eared  nonstutterers  were  higher  than 
chance  and  significant  for  all  temporal  leads  (Column  "All  Ts") . Dichotic 
scores  and  R/L  AAAs  obtained  previously  for  each  subject  are  also  shown 
in  Table  16  for  comparison.  No  correlation  appears  to  exist  between  mean 
dichotic  scores  or  R/L  AAAs  and  mean  percentage  of  correct  classifica- 
tions. 


Percentage  of  Correct  Classification  Scores  at  Left  Hemisphere 
Compared  with  Right--Our  previous  results  would  indicate  that  percentage 
of  classification  scores  would  be  higher  at  left  hemisphere  leads  than 
at  the  right  in  right-eared  subjects.  This  was  not  the  case,  as  can  be 
seen  in  Table  16.  From  the  percentage  of  classification  scores  of  the 
first  group  of  right-eared  nonstutterers,  T3  scores  were  not  consistently 
higher  than  T4  scores,  nor  were  T5  scores  higher  than  T6  scores.  How- 
ever, in  the  next  group,  nonright-eared  nonstutterers,  and  in  nonright- 
eared stutterers,  and  in  nonright-eared  stutterers,  T4  scores  were  as 
high  as  or  higher  than  T3  scores,  which  would  be  expected  if  the  lack  of 
a strong  right  ear  preference  were  an  indication  of  verbal  processing  in 
the  right  hemisphere  rather  than  the  left. 


Percentage  of  Correct  Classification  Scores  of  Right-Eared 
Nonstutterers  Versus  All  Other  Groups--No  significant  differences  existed 
between  percentage  of  classification  scores  of  right-eared  nonstutterers 
and  the  other  groups  (as  there  were  when  V/M  ratios  of  R/L  ratios  were 
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Table  16 


PERCENTAGE  (F  CORRECT  WORD  CLASSIFICATION 
BY  DICHOTIC  GROUPS  AND  ALPHA  ASYMMETRIES* 


s 

T3 

T4 

T5 

T6 

P3 

P4 

All 

Ts 

All  6 

Dichotic 

Scores 

R/L  AAAs 
V/M 

Right 

-Eared 

Nons  tu  t terers 

—!  _ 

LR 

24 

14 

20 

24 

14 

22 

30 

35 

10.5 

0.19 

BL 

22 

34 

28 

24 

16 

14 

30 

27 

11.0 

0.15 

NC 

24 

20 

18 

34 

8 

22 

32 

32 

4.5 

0.075 

MP 

18 

78 

18 

24 

20 

18 

24 

24 

10.0 

0.09 

Mean 

22 

21.5 

21 

26.5 

14.5 

19 

29 

29.5 

9.0 

0.125 

Nonright-Eared  Nonstutterors 

JD 

16 

18 

10 

14 

12 

6 

21 

22 

-1.5 

0.02 

PG 

12 

12 

18 

16 

14 

1 2 

15 

19 

3.0 

-0.08 

JH 

16 

26 

24 

26 

14 

12 

37 

39 

-10.5 

-0.17 

Mean 

14.6 

18.6 

14 

18.6 

13.3 

10.1 

24.3 

26.6 

-3.0 

-0.076 

Right-Eared  Stutterers 

JN 

26 

14 

14 

14 

14 

8 

26 

22 

14.5 

-0.03 

RR 

20 

26 

20 

26 

24 

22 

38 

36 

10.5 

-0.25 

TD 

10 

16 

18 

16 

8 

6 

26 

24 

12.5 

0.13 

BC 

26.2 

24 

16 

24 

8 

18 

29 

32 

18.0 

0.16 

Mean 

20.5 

20 

17 

20 

13.5 

16.8 

29.7 

28.5 

13.8 

0.03 

Nonright-Eared  Stutterers 

DS 

14 

24 

6 

6 

8 

12 

12 

12 

-2 

0.09 

TG 

12 

22 

16 

16 

10 

8 

20 

23 

-1.5 

0.015 

Mean 

13 

23 

11 

11 

9 

10 

16 

17.5 

-1.7 

0.05 

Percentages  of  25  and  above  significant  at  p < 0.05.  Chance  expectation 
is  20%. 


101 


i 


compared)  . Also,  t:he  stutterers  differed  as  a group  from  the  nonstut- 
terers in  their  percentage  of  classification  scores. 

The  only  clear  differences  were  between  JSs  with  strong  ear 
preference  and  Ss  with  no  strong  ear  preference.  All  13  Ss  were  divided 
into  two  groups:  (1)  those  with  dichotic  scores  higher  than  +3  or  lower 

than  -3  (n  = 9)  and  (2)  those  with  dichotic  scores  between  +3  and  -3 
(n  = 4).  WKn  the  ercentage  of  correct  classification  scores  from  all 
six  electrodes  (Table  16)  were  compared  for  Groups  A and  B,  the  scores 
of  Group  A were  found  to  be  significantly  higher  (p  = 0.01,  Mann  Whit- 
ney U)  . When  percentage  of  correct  classification  scores  from  just  the 
four  temporal  electrodes  were  compared  for  Groups  A and  B,  again  the 
scores  of  Group  A were  significantly  higher  (p  = 0.001,  Mann  Whitney  U) . 
This  relationship  can  be  stated  in  an  even  stronger  way:  There  was  a 

positive  rank  correlation  between  strength  of  dichotic  scores  and  per- 
centage of  correct  classification  scores;  that  is,  the  greater  the  dif- 
ference between  the  right  ear  and  the  left  ear,  the  higher  the  percent- 
age of  correct  classification  scores.  This  relationship  is  shown  for 
all  four  temporal  electrodes  (Spearman  Rs  = 0.53,  p < 0.05)  in  Table  17. 
The  rank  correlation  between  dichotic  scores  and  all  six  electrodes 
(i.e.,  incl  ding  parietal  leads)  was  not  significant. 


Percentage  of  Correct  Classification  Versus  T3  V/M  Scores--We 
concluded  in  the  previous  section  that  those  Ss  with  the  lowest  V/M 
alpha  ratios  at  T3  probably  were  engaged  in  relatively  more  verbal  than 
music  processing  at  this  site.  This  conclusion  was  supported  by  the 
findings  that  these  subjects  had  high  right-ear  preferences  on  the  di- 
chotic test,  and  that  those  Ss  with  weak  ear  preference  or  left  ear 
preference  tended  to  have  high  V/M  ratios.  It  is  reasonable  to  hypoth- 
esize that  since  known  speech  centers  produced  a higher  percentage  of 
correct  classification  scores  than  nonspeech  centers  (e.g.,  temporal 
versus  parietal  sites),  Ss  with  low  V/M  ratios  at  T3  might  also  have  had 
a high  percentage  of  correct  classification  scores  at  T3.  That  is,  the 
hemispheric  site  most  engaged  in  language  processing  might  produce  the 
most  discriminab ie  EEG.  Table  18  compares  V/M  T3  scores  and  the  percent 
age  of  correct  classification  scores  for  T3.  There  is  a small  positive, 
but  insignificant,  rank  correlation  between  T3  V/M  scores  and  percentage 
of  correct  classification  scores.  This  suggests  that  the  important  mea- 
sure is  the  difference  between  the  two  hemispheres,  not  the  amount  of 
alpha  change  within  one  hemisphere. 
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Table  17 


RANK  CORRELATION  BETWEEN  PERCENTAGE  OF  CORRECT  EEG  CLASSIFICATION 
(ALL  TEMPORAL  ELECTRODES)  AND  STRONG  DICHOTIC  EAR  PREFERENCE 


Percentage 

Percentage 

Dichotic 

Dichotic 

Correct 

Correct 

_s_ 

Score 

Rank 

All  Ts 

Rank 

TG 

1.5 

1.5 

20 

3.0 

JD 

1.5 

1.5 

21 

4.0 

DS 

2.0 

3.0 

12 

1.0 

PG 

3.0 

4.0 

15 

2.0 

NC 

4.5 

5.0 

32 

il.O 

MP 

10.0 

6.0 

24 

5.0 

LR 

10.5 

8.0 

30 

9.5 

JH 

10.5 

8.0 

37 

12.0 

RR 

10.5 

8.0 

38 

13.0 

BL 

11.0 

10.0 

30 

9.5 

TD 

12.5 

11.0 

26 

6.5 

JN 

14.5 

12.0 

26 

6.5 

BC 

18.0 

13.0 

29 

8.0 

0.53,  p < 0.05 


Percentage  of  Correct  Classification  Scores  Compared  with  Alpha 
Measures--In  the  previous  section,  we  reported  that  there  was  a signif- 
icant (p  < 0.01)  positive  rank  correlation  between  the  extent  of  ear 
preference  and  R/L  verbal  ratios  minus  R/L  music  ratios  in  nonstutterer 
(i.e.,  R/L,  V-M) . Since  these  data  show  a significant  correlation  between 
percentage  of  correct  classification  and  ear  preference,  the  R/L,  V-M 
(T3,  T4)  scores  in  these  Ss  were  examined  to  determine  whether  a signif- 
icant correlation  existed  between  percentage  of  correct  classification 
and  R/L,  V-M  scores.  A Spearman  rank  correlation  between  all  temporal 
percentage  of  correct  classification  scores  and  R/L,  V-M  (T3,  T4)  scores 


resulted  in  an  rg  = 0.60  (0.05  > p > 0.01).  The  same  comparison  between 


percentage  of  classification  scores  of  all  six  electroaes  and  R/L,  V-M 
(T4,  T3)  scores  resulted  in  an  rs  = 0.69  (p  < 0.01). 


Thus,  the  greater  the  differences  in  alpha  activity  between 
hemispheres  (R/L,  V-M)  when  verbal  and  music  tasks  were  compared,  the 
higher  the  classification  scores.  That  is,  Ss  with  more  discriminating 
hemispheres  (greater  response  to  tasks)  have  higher  classification  scores, 
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Table  18 

CORRELATION  BETV/EEN  PERCENTAGE 
OF  CLASSIFICATION  AND  V/M  RATIO  AT  T3 


s 

V/MOT3 

LR 

0.74 

BL 

0.85 

NC 

0.87 

MP 

0.98 

JD 

1.12 

PG 

1.15 

JH 

1.10 

JN 

1.09 

RR 

1.11 

TD 

0.93 

BC 

1.12 

DS 

1.11 

TG 

0.94 

r = 0.23 


Rank 

T3 

Rank 

1 

24 

3.5 

2 

22 

5 

3 

24 

3.5 

4.5 

18 

7 

12 

16 

8.5 

l3 

12 

11.5 

8 

16 

8.5 

6.5 

26 

1.5 

9.5 

20 

6 

4 

10 

13 

11 

26 

1.5 

9.5 

14 

10 

5 

12 

11.5 

Discussion- -The  Ss  of  these  experiments  were  divided  into  the 
four  groups  showing  significant  laterality  differences  according  to  lan- 
guage and  nonlanguage  tasks  (i.e.,  right-eared  nonstutterers,  "o^ight- 
eared  nonstutterers,  right-eared  stutterers,  and  nonright-eared  stutter- 
ers), on  the  logical  grounds  that  these  hemispheric  differences  would 
correlate  with  computer  classification  of  the  EEC  with  speech  production. 
However,  this  correlation  was  not  found,  nor  were  any  significant  cor- 
relations obtained  between  dichotic  listening  scores  (relative  to  h^mi- 
spheric  laterality)  and  "best  classification  hemisphere"  as  obtained  with 
other  subjects.  These  results  might  be  explained  on  the  basis  that  the 
groups  were  too  heterogeneous,  tha  the  N was  too  small,  or  that  the 
tasks  were  not  sufficiently  discriminable . 

A more  plausible  explanation,  however,  is  to  be  found  in  the 
results  showing  a significant  positive  correlation  between  the  amount 
of  ear  preference  and  the  percentage  of  correct  classification  scores. 

If  it  is  assumed  that  the  difference  between  laterality  measures  is  a 
measure  of  cortical  organization  during  speech  tasks  (the  greater  e 
difference,  the  greater  the  cortical  organization  for  language),  correct 
comput Declassification  scores  sboufd  be  expected  to  improve  with  greater 
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cortical  organization  for  language.  This  was  found  and  was  supported 
further  by  the  finding  that,  when  nonlanguage  tasks  were  included  in 
the  ranking  of  cortical  organization,  a higher  percentage  of  computer 
classification  scores  were  obtained. 

Thus,  we  can  conclude  that,  when  there  is  a strong  laterality 
difference  between  language  and  nonlanguage  tasks,  and  when  language  is 
strongly  organized  in  one  hemisphere,  then  the  EEG  will  reveal  more  con- 
sistently a unique  pattern  associated  with  specific  word  production  that 
may  be  recognized  by  a computer. 
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Target  Recognition  and  EEG  Alpha  asymmetry 


Introduction 


The  main  objective  of  the  experiments  reported  here  was  to  determine 
whether  the  pattern  of  arousal  in  the  right  and  left  hemispheres  of  the 
human  brain  influenced  the  efficiency  with  which  perceptual-motor  tasks 
were  executed.  The  study  was  predicated  on  a substantial  number  of 
publications  indicating  that  the  hemispheres  are  specialized  for  dif- 
ferent modes  of  functioning  and  that  cerebral  dominance  in  particular 
situations  can  be  assessed  through  EEG  analysis. 


Marc  Dax  in  1836  and  Paul  Broca  in  1865  (Benton  and  Joynt,  1960) 
noted  that  cerebral  damage  to  the  left  hemisphere  frequently  is  followed 
by  aphasia,  whereas  right  hemisphere  damage  usually  has  no  such  effect. 
More  recent  data  confirm  that  view.  Reversible  chemical  block  of  one 
hemisphere  may  prevent  speech,  whereas  the  other  does  not  (Wada,  1949). 
Damage  or  block  of  the  nonspeech  hemisphere  selectively  impairs  visual 
maze  performance,  face  recognition,  recall  of  geometric  figures,  and  so 
on  (Teuber,  1962;  Milner,  1971).  Disconnection  of  the  two  hemispheres 
through  commissurotomy  revealed  that  the  hemispheres  are  specialized  in 
terms  of  two  general  modes  of  functioning--one  variously  described  as  a 
linear,  analytic,  serial,  logical,  or  propositional  mode;  and  the  other 
as  a nonlinear,  synthetic,  parallel,  apposit iona 1 , or  Gestalt  mode 
(Bogen,  1969a, b;  Bogen  and  Bogen,  1969). 


Electrical  responses  of  the  brain--sensory  evoked  potentials  (EP) , 
contingent  negative  variation  (CNV) , and  the  spontaneous  EEG--also 
indicate  a lateralization  of  function.  Presentation  of  visual  patterns 
representing  words  or  random  dot  designs  produce  different  EP  waveforms, 
and  the  forms  are  most  different  in  the  language  hemisphere  (Buchsbaum 
and  Fedio*  1969,  1970).  Natural  speech  stimuli  produce  larger  EPs  in 
the  language  hemisphere,  especially  in  the  tempo-parietal  region  (Morrell 
and  Salamy,  1971),  and  the  processing  of  speech  can  be  differentiated, 
in  terms  of  EP  waveforms,  from  the  analysis  of  nonspeech  acoustic  param- 
eters (Wood  et  al.,  1971).  The  amount  of  asymmetry  in  the  auditory  EP 
is,  furthermore,  a function  of  the  meaningfulness  of  auditory  stimuli 
as  well  as  of  the  mere  verbal  or  nonverbal  categorization  of  materials 
(Matsumiya  et  al.,  1972). 


The  meaning  also  can  be  in  terms  of  verbs  or  nouns.  Evoked  re- 
sponses to  clicks  that  cue  a verbal  response  differ  as  a function  of  the 


S's  set  to  interpret  the  word  as  a verb  or  noun,  and  the  EPs  associated 
with  verb  sets  are  of  shorter  latency  than  those  in  noun  sets  (Teyler 
et  al.  , 1973).  EPs  produced  by  brief  flashes  are  altered  as  a function 
of  a S' s engagement  in  verbal  or  nonverbal  tasks,  EP  power  in  the  right 
hemisphere  being  largest  during  performance  of  a right  hemisphere  task 
(Galin  and  Ellis,  1975).  Somewhat  complicating  this  literature  are 
several  findings  indicating  that  asymmetric  EPs  are  produced  by  unstruc- 
tured, nonverbal,  nonmeaningful  stimuli  such  as  tones,  light  spots, 
clicks,  and  so  on  (Ruhm,  1971;  Eason  et  al.,  1967;  Gott  and  Boyarsky, 

1972;  Davis  and  Wada,  1974).  The  relationship  of  these  asymmetries  to 
hemispheric  specialization  is  not  entirely  clear  but  may  relate  to  the 
primary  auditory  nature  of  language  and  visuo- spat ia 1 processing  in  the 
nondominant  hemisphere.  Many  of  the  EP  studies  have  been  criticized  as 
methodologically  unsound  (Friedman  et  al. , 1975). 

Oscillatory  EEC  activity  in  the  two  hemispheres  has  been  found  to 
differ  as  a function  of  the  type  of  tasks  in  which  Ss  are  engaged. 

Morgan  et  al.  (1971)  induced  asymmetry  in  the  alpha  band  by  requiring 
Ss  to  respond  to  questions  requiring  either  analytic  (verbal,  mathematical) 
or  synthetic  (visuo-imaginative)  activity.  Although  the  right  hemisphere 
had  more  alpha  activity  in  general  than  did  the  left,  the  proportion  of 
alpha  in  the  right  hemisphere  decreased  during  the  spatial  tasks,  whereas 
that  in  the  left  decreased  during  the  analytic  tasks.  Galin  and  Ornstein 
(1972)  confirmed  Morgan's  findings  using  different  tasks  such  as  writing 
or  mentally  composing  a letter  or  forming  geometrical  designs,  either 
actually  or  imaginatively.  In  a more  detailed  analysis  of  the  Galin  and 
Ornstein  data,  Doyle  et  al.  (1974)  found  that  the  requirement  for  motor 
output  increased  the  degree  of  asymmetry  in  verbal  and  spatial  tasks. 

Morgan  et  al.  (1974)  confirmed  their  previous  findings  by  inducing  EEG 
asymmetry  with  verbal  or  spatial  imaginative  tasks,  and  Butler  and  Glass 
(1974)  observed  relative  alpha  suppression  in  the  left  hemisphere  when 
right-handed  Ss  engaged  in  mental  arithmetic.  Robbins  and  McAdam  (1974) 
also  observed  alpha  asymmetry  when  their  Ss  used  different  cognitive 
modes  in  processing  information,  even  though  the  information  used  (picture 
postcards)  was  the  same  for  each  task.  Only  instructions  to  the  Ss  were 
varied.  Alpha  asymmetry  in  a task  using  both  verbal  and  spatial  imagery 
was  intermediate  to  that  associated  with  use  of  only  one  mode  or  the 
other.  McKee  et  al.  (1973)  showed  that  the  degree  of  asymmetry  was  a 
function  of  task  diff iculty--becoming  greater  with  increasing  difficulty 
of  a linguistic  task.  Herron  (1974)  showed  that  asymmetry  occurred  in 
normal  right-eared  Ss  (defined  with  a dichotic  listening  task)  but  not 
in  Ss  with  no  ear  preference  or  left-ear  dominance.  Stutterers  exhibited 
less  asymmetry  than  nonrials.  Schwartz  et  al.  (1973)  observed  right  alpha 
suppression  when  Ss  whistled  a song  and  left  alpha  suppression  with  reci- 
tation of  lyricc  No  asymmetry  was  observed  during  singing  (a  presumed 
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dual-hemisphere  task).  Increasing  the  difficulty  of  the  tasks  enhanced 
the  degree  of  asymmetry  originally  seen. 


I 


The  foregoing  literature  survey  indicates  that  asymmetric  functioning 
of  the  two  hemispheres  occurs  in  normal  human  subjects  during  performance 
of  different  tasks  and  shows  that  the  asymmetry  can  be  monitored  by  re- 
cording several  types  of  gross  electrical  events.  Because  of  the  ease 
of  EEG  recording,  the  lack  of  necessity  for  time-locking  of  responses, 
and  the  continuous  nature  of  the  EEG,  this  tecnnique  of  electrophysio- 
logical  monitoring  has  the  most  promise  as  a tool  for  determining  the 
role  of  hemispheric  dominance  in  a variety  of  tasks,  so  it  was  used  in 
the  experiment  reported  here. 


Although  several  investigators  have  shown  that  EEG  asymmetry  is 
induced  by  engagement  in  a task,  there  have  been  no  reports  that  the 
degree  of  asymmetry  influences  the  efficiency  with  which  tasks  are 
performed.  If  EEG  asymmetry  is  a functionally  relevant  event,  a rela- 
tionship to  performance  should  be  demonstrable.  Assuming  that  such 
asymmetry  reflects  a direction  of  attention  to  one  or  the  other  proces- 
sing modes,  it  would  be  expected  that  the  perception  of  stimuli  relevant 
to  that  mode  would  be  enhanced,  and  perception  of  irrelevant  stimuli 
would  be  retarded.  Verbal  and  nonverbal  target-recognition  tasks  were 
designed  to  test  that  expectation. 


i 

i 


Experiment  1 - EEG  Asymmetry  and  Performance 
Methods 


Subjects 


Thirty-three  adult  male  or  female  j>s  were  tested  for  handedness 
using  the  following  tests: 


(1)  Handwriting 

(2)  Reaching  for  small  objects 

(3)  Simulated  quick  draw  of  pistol 

(4)  Rifle  aiming 

(5)  Paper  sighting 

(6)  Ball  kicking 

(7)  Family  history 

(8)  Written  questionnaire. 

A score  of  0 was  given  for  left-handed  responses,  1 for  mixed  responses, 
and  2 for  right-handed  responses  on  the  manipulative  tasks.  The  number 


108 


of  right-handed  responses  was  cumulated  from  the  questionnaire  and  added 
to  the  performance  score.  Only  clearly  right-handed  Ss  (those  with 
scores  greater  than  22  out  of  a total  of  29  and  no  family  history  of 
left-handedness)  were  to  be  used  in  the  experiment.  Ninety  percent  of 
right-handed  individuals  have  language  localized  in  the  left  hemisphere. 

Twenty-three  J3s  also  were  given  a preliminary  EEG  screening  to 
familiarize  them  with  the  experimental  environment  and  to  na-row  the  se- 
lection of  Ss.  The  initial  aim  was  to  select  16  j>s , 8 male  and  8 female. 
From  those  23  Ss , the  final  subject  group  was  selected  on  tne  basis  of 
highest  handedness  scores  and  absence  of  EEG  peculiarities  (e.g.,  per- 
sistent alpha  at  a single  lead). 

Task  Development 

Tasks  were  developed  that  presumably  would  differentially 
engage  the  two  hemispheres  and  allow  quantitative  analysis  of  performance. 
A target  detection  design  was  thought  to  be  appropriate  to  these  aims. 

Tne  verbal  task  consisted  of  recognizing  briefly  presented 
words  (50-msec  exposure)  that  were  members  of  a designated  word  category 
(animals,  verbs,  size,  etc.;  see  Table  19).  Similarly,  prememorized 
subsets  of  a general  set  of  spatial  patterns  constituted  nonverbal  tar- 
gets (see  Figure  25).  The  word  and  spatial  tasks  were  run  independently 
in  different  work  periods.  During  the  task,  a stimulus  was  selected 
randomly  and  presented  by  a computer  (Linc-8)  every  1.5  sec,  and  the  S 
pressed  a telegraph  key  as  rapidly  as  possible  when  a target  stimulus 
was  seen. 


Table  19 
WORD  CATEGORIES 


Animals 

Inanimate 

Verbs 

Relations 

Colors 

Size 

Numbers 

Elephant 

House 

Chase 

Husband 

Purple 

Puny 

Unison 

Kitten 

Dirt 

Throw 

Wife 

Green 

Great 

Bimodal 

Hawk 

Desk 

Sing 

Daughter 

Orange 

Small 

Three 

Zebra 

Stove 

Talk 

Cousin 

Yellow 

Medium 

Multiple 
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FIGURE  25  MODERATELY  DIFFICULT  SET  OF  DOT  PATTERNS  USED  IN  THE  SPATIAL  TASK 

Numbers  1-9  were  part  of  a group  of  patterns  |udged  to  be  most  easily  learned 
and  recognized,  Numbers  10-18  were  judged  to  be  moderately  difficult,  and  Numbers 
19  28  were  most  difficult  to  learn  and  recognize. 


Computer  Programs 


Several  programs  were  developed  for  experimental  control  and 
data  analysis.  Initial  programs  were  modified  as  dictated  by  experience 
with  them. 


Early  Programs  for  Experimental  Control 

All  these  programs  were  logically  the  same  but  dealt  with 
different  content--i.e.  , words  and  three  sets  of  patterns  differing  in 
difficulty  of  memorization  and  detection.  These  programs  had  the  fol- 
lowing characteristics  and  functions: 

(1)  Experimental  Control--Program  WORDS  generated  and 
presented  the  list  of  words  incluoed  in  Table  19. 

The  duration  of  exposure,  interstimulus  interval, 
and  number  of  repetitions  of  the  word  list  were 
determined  by  switch  settings  so  were  under  ex- 
perimental control.  Reaction  times  were  obtained 
by  the  computer  and  categorized  in  terms  of  hits 
(correctly  detected  targets)  and  false  alarms 
(nontargets  incorrectly  responded  to).  The 
number  of  target  hits,  target  errors,  nontarget 
hits,  and  nontarget  errors  were  cumulated  and 
printed  out  for  each  stimulus  set  at  the  end  of 
a work  session,  as  shown  in  Table  20.  The  se- 
quence of  stimuli  presented,  their  target  or 
nontarget  status,  and  the  correctness  or  incor- 
rectness of  the  response  to  them  also  was  printed 
(Table  20).  Programs  for  patterns  (GRIDI,  MODGRID, 
and  EASYGRID)  were  logically  the  same  but  differed 
in  difficulty . Only  the  MODGRID  program  has  been 
used  experimentally  so  far. 

A program  called  PRACTICE  was  used  to  present 
patterns  continuously  to  Ss  for  memorization 
before  testing.  Teletype  commands  were  used  to 
select  any  given  pattern  that  was  displayed  until 
another  was  selected.  This  was  incorporated  into 
the  GRID  programs  to  minimize  delay  between  memori- 
zation and  testing. 

Additional  modification  of  these  programs  was  made 
for  use  in  on-line  triggering  of  stimuli.  Presen- 
tation of  a stimulus  was  made  contingent  on  the 
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Taole  20 


PERFORMANCE  DATA 
Example--!  Set,  1 Subject 


SET  2 

TARGET  HITS  - 6 
NONTARGET  HITS  - 22 
TARGET  ERRORS  - 2 
NONTARGET  ERRORS  - 2 
TARGET  HIT  REACTION  TIMES 

379  377  448  507  522  895 

NONTARGET  ERROR  REACTION  TIMES 
333  665 


r 


STIMULUS  SEQUENCE- 

STATUS 

-SCORE 

1401 

301 

801 

701 

101 

2311 

2001 

2311 

401 

2111 

1501 

2211 

2601 

1901 

2700 

1201 

1801 

901 

1301 

2501 

1701 

1101 

2111 

1001 

301 

2410 

1601 

2800 

60.1 

2410 

2211 

201 

— 

Item 

x Target  or 

Correct  or 

Number 

Nontarget 

Incorrect 

presence  of  a signal  on  one  computer  input  that 
was  larger  than  one  of  two  preset  thresholds, 
observable  on  the  Line-scope  as  shown  in  Figure 
26.  Originally,  this  procedure  was  used  in  the 
WORD  and  GRID  programs  separately,  so  that  the 
S observed  only  words  in  one  work  period  and 
patterns  in  another.  The  stimuli  were  also 
triggered  only  by  the  left  hemisphere  in  a given 
session  and  by  the  right  in  another.  Another 
modification  was  made  so  that  either  a word  or 
pattern  was  generated  when  a threshold  was  ex- 
ceeded, and  both  thresholds  could  trigger  a 
stimulus.  The  computer  categorized  reaction 
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FIGURE  26  EXAMPLE  OF  MEAN  SQUARE  (MS)  DIFFERENCE  DISPLAY  WITH  VARIABLE 
UPPER  AND  LOWER  THRESHOLDS 

A MS  difference  exceeding  a threshold  triggered  a stimulus. 
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time  into  four  bins  in  terms  of  the  stimulus 
and  triggering  hemisphere--i  .e . , word  left, 
word  right,  pattern  left,  pattern  right. 

Another  version  of  the  foregoing  program  was 
designed  for  use  in  a biofeedback  paradigm 
for  learning  autocontrol  of  EEG  asymmetry. 
Words  occurred  when  the  upper  threshold  was 
exceeded,  and  patterns  occurred  when  the  lower- 
threshold  was  exceeded . Either  left  or  right 
hemisphere  alpha  could  be  made  to  trigger 
words  or  patterns  by  changing  the  input 
polarity. 

(2)  Data  Analysis--Programs  were  written  to 

facilitate  spectral  analysis  of  EEGt  and  to 
obtain  quantification  of  spectral  parameters. 
One  program,  COOLEY,  performed  spectral  analy- 
sis of  sequential  blocks  of  digitized  EEGs  and 
stored  the  spectra  on  the  second  half  of  Line 
tapes.  Another  program,  AVETYPE,  obtained  and 
printed  several  parameters  of  the  spectra,  as 
indicated  in  Table  21.  The  several  parameters 
were: 

(a)  Digital  representations  of  the  value 
of  each  spectrum  point. 

(b)  Average  power  (A)  = The  average  value 
of  the  points  lying  between  preset 
limits  (frequency  band) . 

(c)  Peak  power  (P)  = The  largest  value  in 
the  defined  frequency  band. 

(d)  Peak  position  (PP)  = The  ordinal  posi- 
tion of  the  largest  value,  an  indication 
of  the  dominant  frequency  in  the  band. 

(e)  FC  = Frequency  centroid,  another  measure 
of  dominant  frequency  or  the  "balance 
point"  of  the  spectrum  in  the  frequency 
dimension. 


FC 
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EXAMPLE  OF  COMPUTER  QUANTIFICATION  OF  EEG  SPECTRA 
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where  n^  = ordinal  position  and  = the 
spectrum  value  at  position  n^  If  all 

X s = 1,  FC  = the  mean  of  all  n . 

1 i 

(f)  P/A  = An  indication  of  the  amount  of  dis- 
persion in  the  band. 

(g)  PP/A  = Ratio  of  frequency  and  power,  an 
indication  of  EEG  activation  (the  activa- 
tion index,  AI),  assuming  that  frequency 
is  directly  proportional  to  arousal. 
Redundant  measures  of  this  are  obtained 
using  other  measures  (PP/P,  FC/A,  and 
AC/P)  to  determine  the  most  sensitive 
measure . 


Instrumentation 

Ss  were  tested  in  a small,  screened  enclosure.  Beckman 
sintered  Ag-AgCl  electrodes  were  used  to  conduct  EEG  signals  to  a 
Beckman  Type  R Dynagraph.  Amplified  signals  were  recorded  on  an  Ampex 
SP-300  instrumentation  recorder.  Linc-8  computer-generated  relay 
closures  operated  Grass  S-4  stimulators  that  provided  a stimulus  marker 
on  polygraph  paper  and  a synchronization  pulse  on  tape  to  control  data 
playback.  Spectral  plots  were  made  with  a Hewlett-Packard  1024  X-Y 
plotter.  Calibration  sine-waves  were  produced  by  a Hewlett-Packard 
signal  generator.  On-line  studies  included  two  Ballantine  Model  350 
RMS  meters  and  General  Radio  1952  active  electronic  filters. 


Procedures 


Electrodes  were  placed  over  parietal  cortex  between  scalp  loca- 
tions P3  and  T5  and  P4  and  T6  of  the  International  10/20  System,  near 
Wernike  s speech  area,  and  at  01  and  02,  visual  receiving  areas.  Resis- 
tance was  checked  to  be  below  lOkft  and  was  usually  5H2  or  less.  Ss  were 
instructed  in  the  general  nature  of  the  experiment  during  placement  of 
electrodes  and  then  were  shown  examples  of  the  word  and  pattern  stimuli 
as  they  would  appear  during  the  experiment.  This  procedure  ensured  that 
the  S could  perceive  the  stimuli  clearly.  Pretest  memorization  of  pat- 
terns involved  pseudorandom  presentation  of  four  selected  patterns  until 
the  S indicated  he  knew  them  well  (this  always  involved  four  presenta- 
tions of  each  pattern). 
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During  testing,  each  of  the  4 target  stimuli  were  presented 
twice,  so  that  there  was  a total  of  32  stimuli  in  each  performance  set 
(28  stimuli,  4 of  them  presented  twice),  and  8 (25%)  of  them  were  targets. 
Each  set  was  presented  twice  in  succession  so  the  j3  received  64  consecu- 
tive stimuli,  defining  a work  period.  Two  work  periods  for  words  and 
two  for  patterns  were  given  each  _S.  The  order  of  word  and  pattern  tasks 
were  balanced  across  _Ss.  A control  task  (C)  in  which  either  words  or 
patterns  occurred  also  was  included.  This  task  was  expected  to  produce 
EEC  symmetry  and  provide  a baseline  task  for  comparative  purposes.  This 
task  always  intervened  between  the  first  and  last  work  sessions  for  the 
other  tasks  as  follows: 


Task  Order 


1 W1W2-P1P2-C1C2CA-P3P4-W3W4 

2 p1p2.«1«2.c1c2c3c4,w3w4.p:)p4 


Approximately  4-min  breaks  were  given  between  work  periods 
while  data  were  printed  by  the  computer. 


Before  testing  was  begun  for  each  3,  a 10-Hz  sine  wave  was 
introduced  into  each  polygraph  and  tape  recorder  channel,  and  all  gains 
were  equalized.  The  first  step  in  data  analysis  was  analog- to-digital 
conversion  of  the  stored  EEGs.  Before  digitizing,  the  recorded  calibra- 
tion sine  waves  were  used  to  equalize  gain  levels  to  the  several  computer 
input3.  Sine  waves  from  each  channel  were  set  to  ± 200  octal  display 
points  on  the  computer  by  means  of  the  CALIBRAT  program.  EEG  epochs  of 
4 sec  each,  based  on  256  points  and  16-msec  sampling  intervals  were  ob- 
tained after  a fixed  delay  from  each  synch  pulse  so  that  the  EEG  sample 

overlapped  Trials  3 and  4,  9 and  10 33  and  34,  and  so  on.  Five 

EEG  epochs,  therefore,  were  obtained  in  association  with  each  performance 
set  (32  trials).  When  spectral  analysis  was  completed  with  the  COOLEY 
program,  the  five  spectra  associated  with  each  performance  set  were 
averaged  together  so  a single  spectrum  formed  the  basic  EEG  data  for 
each  performance  set-. 
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To  define  each  S's  alpha  band,  all  of  each  S's  spectra  that 
evidenced  alpha  were  averaged  together,  stored  on  Line  tape,  and  plotted. 
The  plots  were  studied  to  determine  at  what  frequency  alpha  activity  be- 
gan to  appear  and  diminish.  Positions  on  the  spectra  that  were  judged 
to  delimit  alpha  were  marked  (Figure  27)  , and  the  octal  numbers  repre- 
senting the  horizontal  positions  were  obtained  with  a cursor  program  on 
the  Linc-8.  If  alpha  was  not  clearly  present,  the  whole  8-  to  13-Hz 
band  was  used.  This  procedure  seemed  more  realistic  than  arbitrarily 
applying  fixed  limits  to  all  Ss.  Clearly,  individual  differences  in 
EEG  spectra  exist.  Some  Ss  have  very  consistent  and  narrow  alpha  bands, 
whereas  others  have  wide  bands,  end  some  Ss  have  alpha  of  different 
center  frequencies  than  others.  Spectrum  values  were  obtained  with 
AVETYPE  and  analyzed  in  terms  of  hemispheric  asymmetry  associated  with 
the  tasks  and  their  relationships  to  performance  efficiency. 


Results 


After  8 5>s  were  run,  preliminary  analyses  were  undertaken  to  deter- 
mine whether  meaningful  data  were  being  generated.  First,  differences 
in  general  hemispheric  activity  were  studied  as  a function  of  the  tasks. 
We  expected  that  the  relative  amount  of  alpha  activity  in  the  right 
hemisphere  would  be  less  in  the  spatial  task  than  in  the  verbal  task. 

Figure  28  shows  the  relative  amounts  of  average  alpha  power  (ex- 
pressed in  arbitrary  units)  in  the  hemispheres  during  performance  in 
the  two  tasks.  In  both  verbal  and  spatial  tasks,  average  alpha  power 
was  greater  in  the  right  than  in  left  hemisphere  (Dv  = 12.63,  t = 3.0, 
df  = 7,  p < 0.02;  Ds  « 20.13,  t = 4.41,  df  = 7,  p < 0.02;  two  tailed). 
There  was  no  significant  difference  in  the  right  over  left  (R/L)  ratios 
for  verbal  and  spatial  tasks,  but  in  terms  of  right-left  difference 
scores,  the  verbal  and  spatial  comparison  approached  significance 
(D  = 7.5,  t ■ 2.65,  df  «*  7,  p < 0.05).  Thus,  there  was  a general  bias 
in  terms  of  greater  right  hemisphere  alpha  power  in  the  parietal  region, 
and  the  suggestion  of  enhancement  of  that  bias  in  the  spatial  task. 

The  apparent  differences  in  the  occipital  region  shown  in  Figure  28 
were  not  statistically  reliable. 

Alpha  frequency  in  t'ie  two  hemispheres  and  tasks  as  measured  by 
peak  position  is  shown  in  Figure  29.  The  pattern  of  mean  frequencies 
across  hemispheres  and  tasks  is  as  would  be  expected — i.e.,  higher  fre- 
quency in  the  left  than  the  right  during  verbal  processing  and  vice  versa 
for  spatial  processing.  However,  the  Task  X Placement  interaction  was 
not  statistically  significant,  and  this  pattern  was  not  duplicated  when 


118 


FREQUENCY 


FIGURE  27  AVERAGE  EEG  FREQUENCY  SPECTRA  FROM  THREE  SUBJECTS  SHOWING 
INDIVIDUAL  DIFFERENCES  OF  SPECTRAL  SHAPES  !N  THE  ALPHA  BAND 
AND  THE  LIMITS  SET  FOR  ALPHA  ACTIVITY  IN  THESE  SUBJECTS 


VERBAL  SPATIAL  VERBAL  SPATIAL 


ELECTRODE  PLACEMENTS  AND  TASKS 

I 

I 

FIGUPE  28  AVERAGE  PARIETAL  (P3,  P4)  AND  OCCIPITAL  (01.  02)  ALPHA  POWER 
IN  VERBAL  AND  SPATIAL  TASKS 


AVERAGE  FREQUENCY  (peak  position) 


14  h 


12  - 
10  - 
8 - 


4 - 
2 - 
0 - 


P3  P4  P3  P4 

VERBAL  SPATIAL  VERBAL  SPATIAL 

ELECTRODE  PLACEMENTS  AND  TASKS 


FIGURE  29  AVERAGE  PARIETAL  <P3,  P4)  AND  OCCIPITAL  (01,  02)  ALPHA  FREQUENCY 
(PEAK  POSITION)  IN  VERBAL  AND  SPATIAL  TASKS 


the  frequency  centroid  measure  was  used.  There  were  essentially  no 
differences  at  the  occipital  sites  (Figure  29). 

Although  contrary  to  the  expected  direction  of  induced  alpha  power 
asymmetry,  the  parietal  difference  in  verbal  and  spatial  tasks  (enhanced 
alpha  in  the  right  hemisphere  in  the  spatial  task)  was  the  same  in  seven 

of  the  eight  Ss,  so  study  of  performance  relationships  to  asymmetry  w <s 
pursued. 

Detection  performance  was  excellent  in  both  tasks,  averaging  807. 
but  it  was  slightly  better  in  the  verbal  task.  An  average  of  6.91  hits 
of  8 targets  occurred  in  the  verbal  task,  and  5.88  in  the  spatial  task 
That  difference  was  significant  (5  - 1.03,  t = 2.45,  df  = 63,  p < 0.02). 
Average  reaction  times  for  correctly  detected  targets  (target  hits)  for 
each  performance  set  were  first  computed  on  the  basis  of  all  scores 
except  a few  anticipatory  responses  of  1 to  6 msec.  Means  and  standard 
deviations  (SDs)  were  obtained,  and  then  all  RTs  greater  than  900  msec 
that  exceeded  the  mean  by  1.5  SDs  were  excluded,  and  the  mean  was  re- 
computed. This  procedure  eliminated  extremely  long  RTs  that  would 

distort  the  mean.  Mean  RTs  for  each  performance  set  are  shovm  in  Table  22 
and  plotted  in  Figure  30A. 

To  study  asymmetry  and  performance,  right-over- left  hemisphere 
ratios  for  alpha  power,  frequency  (peak  position)  and  an  Activation 
Index  (AI)  were  computed  (see  Table  22  for  means  and  SDs).  The  AI 
score  is  the  ratio  of  frequency  over  power  (AI  - f/p).  Since  frequency 
is  directly  related  to  cortical  arousal,  and  power  is  inversely  related 
to  it,  the  AI  score  should  represent  the  cortical  state  more  sensitively 
than  either  frequency  or  power  alone.  These  scores  were  ranked  from  low 
to  high  for  each  S over  the  four  performance  blocks.  Average  RTs  to 
correctiy  detected  targets  for  each  set  then  were  arranged  in  accordance 
with  the  R/L  ratios-i.e.,  the  following  reordering  was  done  for  each  S: 


R/L  Ratio 
RT 


Original  Order  Alpha 
Power  Ratio  and  RT 
Performance  Set 


0.79  1.19  0.64  0.59 

549  521  405  392 


Reordered  Ratios 
and  RT 
R/L  Ratio 


Low 

Med. 

Mod . 

High 

0.59 

0.64 

0.79 

1.19 

392 

405 

549 

521 

Means  and  standard  deviations  of  RT  ordered  according  to  the  three 
asymmetry  measures  are  shown  in  Table  23.  Figure  30B  shows  mean  RTs  as 
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MEAN  REACTION  TIMES  AND  R/L  RATIO  SCORES  IN  FOUR  PERFORMANCE  SETS 
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0.69  0.30  0.80  0.64  0.71  2.64 


MEAN  REACTION  TIME  (msec)  MEAN  REACTION  TIME  (msec) 


R/L  RATIO  ALPHA  FREQUENCY  (pp)  R/L  RATIO  ACTIVATION  INDEX 

—RIGHT  HEMISPHERE  AROUSAL-  —RIGHT  HEMISPHERE  AROUSAL- 


FIGURE  30  REACTION  TIME  AS  A FUNCTION  OF  (A)  PERFORMANCE  SET,  (B)  R/L 
AVERAGE  POWER,  (C)  R/L  ALPHA  FREQUENCY,  (01  R/L  ACTIVATION 
INDEX  = FREQUENCY/POWER 


MEANS  AND  STANDARD  DEVIATIONS  OF  RT 
AS  A FUNCTION  OF  R/L  ASYMMETRY  SCORES 


VerbaJ 


Low 

Med . 

Mod. 

Average 

X 

573.  1 

530.5 

529.9 

power 

SD 

166.6 

90.4 

119.2 

Peak 

X 

487.0 

557.3 

526.1 

position 

SD 

83.1 

101.9 

85.7 

Activation 

X 

484.0 

505.5 

563.5 

index 

SD 

85. 9 

77.4 

120.6 

Spatial 


High 

Low 

Med. 

Mod. 

High 

494.6 

86.4 

649.6 

77.2 

659.3 

94.9 

670.8 

72.8 

671.0 

51.9 

565.8 

179.3 

708.3 

73.5 

678.5 

69.0 

624.6 

60.8 

639.3 

67.3 

583.1 

160.4 

680.9 

50.3 

700.4 

84.3 

638.5 

69.7 

630.9 

70.9 

a function  of  ordered  average  power  ratios.  As  right  hemisphere  power 
ircreased , there  was  a slight  tendency  for  RTs  to  patterns  to  increase 
and  a stronger  tendency  for  RTs  to  words  to  decrease.  In  Figures  30C 
and  D,  the  opposite  trends  were  apparent  as  a function  of  alpha  frequency 
and  AI  ratios. 

Neither  the  Task  X Ratio  interaction  in  a 2 X 2 X Ss  analysis  of 
variance  on  low  and  high  ratio  levels  nor  low  versus  high  t comparisons 
were  significant  for  average  power.  The  Task  X Ratio  interaction  in  a 
2 X 2 X Ss  analysis  approached  significance  for  the  frequency  measure 
(F  = 4.32,  df  = 1.7,  0.05  < p < 0.10).  A paired  t comparison  on  RT  at 
low-  and  high-frequency  ratios  was  significant  for  the  spatial  task  but 
not  for  the  verbal  (D  * 69,  t = 2.28,  df  * /,  p < 0.05,  one-tailed  test). 
A 4 X 3 X Ss  analysis  of  variance  on  the  AI  scores  showed  a significant 
Task  X Ratio  interaction  (F  = 4.15,  df  = 3.21,  p < 0.025).  This  inter- 
action was  due  primarily  to  a significant  effect  in  the  verbal  task 
where  the  extreme  scores  were  reliably  different  (T>  = 99.13,  t - 2.06, 
df  “ 7,  p < 0.05  one-tailed  test),  although  there  was  a contributing 
50-msec  difference  in  the  spatial  task  in  the  opposite  direction.  In 
all  these  analyses,  there  were  significant  effects  of  task  (e.g.,  for 
the  AI  analysis  F = 29,  df  » 1.7,  p < 0.005),  RT  being  slower  in 
the  spatial  task. 

► 

The  large  difference  (100  msec)  in  RT  between  verbal  performance 
Sets  2 and  3 (2  having  faster  RTs)  provided  another  way  of  observing 
performance  and  asymmetry  relationships--i.e . , those  two  sets  were 
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compared  in  terms  of  EEG  asymmetry  to  determine  whether  asymmetry  scores 
differed  between  them.  For  the  power  measure,  6 of  the  8 Ss  had  larger 
R/L  ratios  in  Set  2 than  in  Set  3— i.e.,  greater  left  hemisphere  arousal 
associated  with  fast  RTs  to  words.  The  magnitudes  of  the  scores  were 
highly  variable,  however,  and  the  mean  difference  was  not  significant. 

The  frequency  and  AI  scores  were  significantly  different  in  the  two  sets, 
especially  the  latter  score--7  of  the  8 Ss  had  smaller  R/L  ratios  in  the* 
second  than  in  the  third  sets  (Dpp  = 0.24,  t = 2.11,  df  = 7,  p < 0.05  one 
tailed  test;  DAI  = 0.65,  t = 2.74,  df  = 7,  p < 0.025  one-tailed  test)-- 
i.e.,  greater  left  hemisphere  arousal  associated  with  fast  RTs  to  words. 


Discussion 


Because  unexpected  enhancement  of  right  hemisphere  alpha  by  the 
spatial  task  was  observed  during  preliminary  examination  of  the  data 
after  only  half  the  planned  Ss  were  run,  more  thorough  analyses  of  th 
data  were  thought  to  be  necessary.  The  paradoxical  enhancement  occuired 
in  7 of  8 Ss  and  could  not  be  explained  by  misplacement  of  electrodes 
or  other  technical  errors.  Therefore,  performance  relationships  were 
determined  to  test  the  adequacy  of  the  general  paradigm.  Because  of 
the  small  number  of  Ss,  these  data  were  construed  to  provide  a prelimi- 
nary test,  rather  than  definitive  data,  but  it  seemed  prudent  to  undertake 
the  analyses  rather  than  continue  to  run  Ss  in  a possibly  inadequate 
paradigm. 


Analysis  of  Standard  Alpha  Band 


To  check  the  assumption  derived  from  the  previous  data  that  it 
was  advantageous  to  establish  alpha  bands  on  an  individual  basis,  we 
reanalyzed  the  data  using  i standard  9-  to  12-Hz  band  for  each  S. 

Figure  31  shows  a plot  of  RT  as  a function  of  alpha  power  ratios  using 
either  individual  or  standard  definition  of  alpha.  There  are  no  differ- 
ences in  the  plots  indicating  that,  for  the  purposes  of  studying  asymmetry- 
performance  relationships,  the  standard  measure  is  adequate. 


Evoked  Potentials  to  Correctly  Detected  Targets 

When  words  and  patterns  are  flashed  separately  to  right  and 
left  visual  fields,  differences  in  evoked  potentials  (EPs)  are  evident 
(Buchsbaum  and  Fedio,  1969).  In  this  experiment,  both  fields  were  stimu 
lated,  but,  theoretically,  the  hemispheres  specialized  for  verbal  and 
nonverbal  processing  should  still  evidence  difference  in  the  EPs  to  word 
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and  pattern  stimuli.  In  these  results,  the  EPs  were  not  very  clear  to 
words,  and  no  obvious  differences  existed  between  the  hemispheres. 

However,  potentials  evoked  by  pattern  stimuli,  primarily  the  first  nega- 
tive component,  were  usually  (in  5 out  of  the  8 Ss--TB,  RS , JF,  RS,  MW) 
slightly  larger  in  the  right  hemisphere  than  in  the  left  (Figure  32). 
Excluding  the  reversed  S,  there  was  an  average  amplitude  difference 
(base  to  peak)  of  the  major  negative  component  of  4.3  uV,  the  component 
being  larger  in  the  right  hemisphere  (t  = 2.5,  df  = 6,  p < 0.05,  twn- 
tailed  test) . 

The  EP  reversal  in  jS  PT  brought  attention  to  her  specific 
pattern  of  EEG  asymmetry-performance  relationships,  since  she  possibly 
could  have  reversed  or  mixed  dominance.  With  respect  to  RT  performance, 
this  S was  reversed  in  terms  of  average  power,  frequency,  and  the  AI 
measure  (i.e.  , her  RT  to  words  became  faster  with  increasing  right  hemi- 
sphere arousal  in  contrast  to  the  average  trend  across  Ss).  She  was  also 
the  only  JS  with  less  right  hemisphere  alpha  in  general  during  the  verbal 
task,  and  she  had  the  largest  R/L  alpha  ratio  in  the  occipital  region 
opposite  the  mean  trend.  Of  15  such  measures  studied,  this  S showed 
nonconformity  with  mean  trends  on  9 of  them.  The  most  nonconformities 
shown  by  any  other  S was  five  (Figure  33). 

As  a further  check  of  PT 1 s state  of  laterality,  Jeannine  Herron 
of  Langely  Porter  Institute,  San  Francisco,  CA,  examined  her  in  a dichotic 
listening  task.  Although  a slight  right-ear  advantage  was  apparent  in  the 
task  (a  score  of  +5) , the  absolute  score  was  small  compared  with  the  mean 
of  other  Ss  similarly  tested  in  Dr.  Herron's  laboratory  (+11),  indicating, 
perhaps,  a mixed  dominance.  Assuming  that  this  might  constitute  one 
of  the  approximately  107„  of  right  handers  with  opposite  dominance,  her 
performance  data  were  reversed  to  determine  the  effect  on  significance 
levels  for  the  average  power  score.  The  original  and  revised  plots  are 
shown  in  Figure  34.  There  was  no  effect  on  the  data  for  nonverbal  per- 
formance, but  the  group  curve  for  verbal  performance  was  linearized,  and 
the  extremes  became  significantly  different  (t  = 2.4,  p < 0.025). 

The  extent  to  which  the  original  or  revised  scores  in  this  S 
are  taken  to  represent  the  real  state  of  affairs  in  the  above  depends  on 
the  criteria  for  hemispheric  dominance  considered  most  exact.  Handedness 
is  fairly  reliable,  but  a known  proportion  of  dextrals  do  have  right 
hemisphere  dominance.  The  dichotic  listening  test  was  ambiguous.  Indices 
most  directly  reflecting  the  perceptual  processes  under  study  (e.g., 
the  evoked  potentials)  might  be  the  best  indicator  of  where  in  the  ner- 
vous system  information  is  preferentially  treated.  In  the  least,  these 
results  indicate  that  more  exact  procedures  for  selecting  dominance  need 
to  be  used  in  subsequent  studies,  or,  perhaps,  that  the  EP  be  adopted 
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arbitrarily  as  the  deciding  criterion  in  cases  of  ambiguity.  Ulr  unately , 
the  general  procedure  used  here  might  be  a very  sensitive  one  for  deter- 
mining cerebral  dominance. 

The  data  presented  above  indicate  that  EEG  asymmetry  does 
influence  performance  in  the  perceptual-motor  task  employed  here. 

Whereas,  statistically,  the  individual  comparisons  are  weak,  the  dif- 
ferential profiles  of  RT  scores  as  a function  of  task  and  asymmetry  are 
complementary  and  of  opposite  directions  with  different  scores  (e.g., 
power  and  frequency)  in  concordance  with  the  normal  interpretation  of 
the  functional  significance  of  those  scores.  The  fact  that  the  Task  X 
Ratio  profiles  are  opposite  for  frequency  and  power  would  itself  suggest 
a real  effect  even  if  there  were  no  statistically  reliable  effects  using 
each  score  alone.  The  AI  score  takes  advantage  of  the  apparently  opposite 
relationships  of  frequency  and  power  to  cortical  arousal  and  shows  sta- 
tistically reliable  effects  on  RT  performance.  In  addition,  it  could 
be  argued  that  the  significantly  slower  RT  in  the  spatial  task  in  general 
was  related  to  the  overall  enhanced  alpha  in  that  task.  The  results  also 
indicate  that  greater  attention  should  be  paid  to  frequency  differences 
in  the  hemispheres  rather  than  studying  power  alone. 

From  a strictly  scientific  view,  the  findings  above  must  be 
repeated  with  a larger  number  of  Ss  to  establish  greater  confidence  in 
them.  However,  given  the  program  goal  to  study  man-machine  interaction, 
the  data  seemed  suggestive  enough  in  a practical  sense  to  attempt  to 
determine  the  influence  of  momentary  interhemispheric  asymmetries  on 
perception  in  an  on-line  paradigm. 


Experiment  2 - On-Line  Trigger  of  Words  and  Patterns  by  EEG  Alpha 
Power  Asymmetry 

Since  the  first  experiment  suggested  that  average  EEG  asymmetries 
over  a performance  set  influenced  RT , it  was  predicted  that  if  momentary 
asymmetries  in  alpha  power  were  used  to  trigger  the  word  and  pattern 
stimuli,  a similar  finding  would  be  obtained — i.e.,  that  RTs  to  words 
triggered  by  a left  hemisphere  alpha  burst  would  be  slower  than  when 
words  were  triggered  by  right  hemisphere  alpha  bursts.  The  opposite 
would  be  expected  for  patterns.  More  generally,  opposite  results  should 
be  obtained  for  words  and  patterns,  whatever  the  particular  results  with 
one  of  the  tasks. 
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Methods 


The  same  general  methodology  as  used  before  was  employed  in  this 
experiment , except  that  stimuli  were  triggered  by  alpha  asymmetries 
rather  than  automatically  by  computer.  One  effect  of  this  was  to  intro- 
duce a variable  interstimulus  interval.  Figure  35  indicates  the  method 
used  to  obtain  on-line  alpha  power  asymmetries.  EEGs  from  left  and 
right  parietal  areas  were  routed  to  General  Radio  1952  active  filters 
set  to  9-  to  12-Hz.  Filter  outputs  were  connected  to  two  Ballantine  320 
RMS  meters  that  were  recorded  on  two  polygraph  channels.  A differential 
measurement  of  the  MS  outputs  on  the  polygraph  provided  an  indication  of 
on-line  momentary  asymmetries.  Relatively  large  right  hemisphere  alpha 
activity  produced  an  upward  pen  deflection,  and  left  hemisphere  alpha 
caused  a downward  deflection.  This  channel  was  input  to  the  Linc-8 
computer.  Upper  and  lower  limit  thresholds,  visible  on  the  scope,  could 
be  set  to  produce  stimuli  triggered  by  either  hemisphere  alone  or  by  both 
hemispheres.  One  program  generated  words  and  another  patterns,  so  that 
a S performed  in  the  word  task  and  pattern  task  during  different  work 
periods.  The  order  of  word  or  pattern  sets  was  counterbalanced  across 
Ss  with  a balanced  latin  square  design.  Two  word  am'  two  pattern  sets 
of  64  stimuli  each  were  given  each  S,  one  with  right  hemisphere  alpha 
trigger  and  one  with  left  trigger. 


Results 


Figure  36  shows  a sample  polygraph  record  f^m  one  S.  Because  the 
task  becomes  quite  boring  if  stimuli  are  presented  very  slowly,  and 
because  Ss  are  rather  alert  and  thus  are  not  producing  a great  deal  of 
alpha,  the  trigger  thresholds  had  to  be  set  very  low  to  maintain  an  ade- 
quate stimulus  presentation  rate,  and  stimuli  were  very  often  triggered 
by  trivial  differences  in  alpha  asymmetry.  Because  of  this  problem,  not 
all  RT  scores  would  represent  an  event  associated  with  clear  asymmetry. 
Therefore,  the  magnitude  (in  millimeters  of  pen  deflection)  of  each 
triggering  asymmetry  was  determined  for  each  target  hit,  and  the  ten 
largest  asymmetries  for  right  and  left  hemisphere  triggers  were  obtained. 
Mean  RTs  were  computed  from  those  sets  of  ten  trials  resulting  in  four 
mean  RTs  for  each  S-RTs  to  right-  and  lef t-hemisphere-triggered  words 
(WR  and  WL) , and  right-  and  left-hemisphere-triggered  patterns  (PR  and 
PL),  which  are  si  jwn  in  Table  24. 

Comparisons  of  RTs  to  right-  and  left-triggered  stimuli  across  Ss 
were  not  significant  in  either  task.  In  other  words,  Ss  were  not  con- 
sistent in  showing  slower  RTs  to  left-triggered  words,  right-triggered 
patterns,  and  so  on.  However,  arrangement  of  the  means  in  a 2 X 2 
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EXAMPLE  OF  BILATERAL  EEG  TRACINGS,  THEIR  MEAN  SQUARES,  AND  THE 
DIFFERENCE  OF  MEAN  SQUARE  VOLTAGE  IN  THE  TWO  HEMISPHERES 


FIGURE  36 


Table  24 


MEAN  RT  AS  A FUNCTION  OF  TASK  AND  HEMISPHERE 
DURING  ON-LINE  TRIGGERING  OF  STIMULI 
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BB 
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12.6 
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MW 
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DH 
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511.40 

496.50 

54.9 

14.9 

RB 
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684.20 

660.0 

70.5 
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•ff 

The  hemisphere  condition  used  as  the  minuend  in  the 
verbal  task  (always  the  one  with  the  largest  RT)  also 
was  used  as  the  minuend  in  the  spatial  task. 


contingency  table  (Table  25)  revealed  a pattern  of  distribution  not  com- 
pletely attributable  to  chance  (x^  = 4.66,  0.05  < p < 0.10) , suggesting 
that  the  asymmetry-performance  relationships  are  different  for  words  and 
patterns.  In  developing  this  table  Ss  first  were  categorized  in  terms 
of  their  mean  RTs  to  right-  or  left-triggered  words--i.e. , if  WR  was 
greater  or  lesser  than  WL.  Given  the  validity  of  the  original  hypothesis, 
it  would  be  expected  that,  in  all  Ss , RT  in  the  WR  condition  should  be 
less  than  in  the  WL  condition  and  that  RTs  in  the  PR  condition  would  be 
greater  than  in  the  PL  condition.  All  Ss  then  would  be  categorized  as 
WR  < WL  and  PR  > PL  (Quadrant  B)  in  the  2 X 2 contingency  table.  Em- 
pirically, however,  that  distribution  was  not  obtained.  Rather,  only 
four  Ss  fell  in  that  quadrant,  whereas  five  were  in  Quadrant  C (WR  > WL 
and  PR  < PL)  and  one  each  was  in  the  other  two.  That  distribution  is 
almost  different  than  expected  by  chance  in  terms  of  x (0.05  < p < 0.10) 
and  is  meaningful  in  terms  of  hemispheric  specialization  in  that  the 
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Table  25 


NUMBER  OF  SUBJECTS  WITH  RT  PATTERNS  DEFINED 
BY  THE  FOUR  QUADRANTS 


WR  < WL 


WR  > L’L 


PR  < PL  PR  > PL 
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1 


C 


5 


B 


4 


D 
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pattern  of  hemispheric  arousal  associated  with  fastest  RTs  was  opposite 
for  the  two  tasks.  j5s  who  had  fastest  RTs  to  words  with  right  alpha 
trigger  had  slowest  RTs  to  patterns  with  right  alpha  trigger  and  vice 
versa  for  left  triggers. 

2 

The  x test  with  which  a marginally  significant  effect  was  found  in 
these  data  is  rather  insensitive  and  does  not  consider  magnitudes  of 
differences.  To  examine  these  data  more  fully,  we  compared  RT  difference 
scores  in  the  tasks  and  hemispheres  using  a contingent  method  of  sub- 
traction in  the  spatial  task,  dependent  on  direction  of  the  verbal  effect. 
That  is,  the  smallest  average  RT  in  the  verbal  tasr  associated  with  WR 
or  WL  trigger  conditions  always  was  subtracted  from  the  largest  so  that 
these  difference  scores  were  always  positive  (e.g.,  for  S 1,  RTWL  = 485.2 
and  RT^  = 475.10  for  a difference  of  +29.1;  for  S 3,  RTWR  = 502.2  and 
rtWL  = 454-7  for  a difference  of  +47.5).  To  obtain  the  spatial  difference 
score,  the  trigger  condition  (right  or  left)  associated  with  the  largest 
vc^baL  RT  (minuend)  also  always  constituted  the  minuend  in  obtaining  the 
difference  score  for  the  spatial  task.  If  no  consistent  difference 
existed  between  the  direction  of  effect  in  the  verbal  task  and  the  spatial 
task,  no  consistent  algebraic  sign  would  be  associated  with  the  differ- 
ences obtained  in  the  spatial  task,  and  no  significant  difference  would 
occur  in  the  means  of  the  verbal  and  spatial  difference  columns.  These 
subtractive  procedures  ajid  results  also  are  tabulated  in  Table  24.  The 
effect  was  significant  (D  = 65.5,  t = 539,  df  = 10,  p < 0.01,  two-tailed 
test).  Although  right  hemisphere  alpha  triggers  were  not  invariably 
associated  with  slower  RTs  to  patterns  and  faster  RTs  to  words  (and  vice 
versa  for  left  alpha  triggers),  these  results  indicate  clearly  thaL  the 
two  hemispheres  have  opposite  relationships  to  particular  tasks.  The 
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results  also  support  the  general  conclusion  of  the  first  experiment 
that  EEG  alpha  asymmetry  is  related  to  perceptual-motor  performance. 

It  is  not  clear  from  the  second  experiment  whether  cerebral  organi- 
zation of  the  subsets  of  Ss  actually  differed  or  whether  some  pecularities 
of  the  triggering  system  resulted  in  opposite  EEG-performance  relation- 
ships in  the  two  subsets.  Mean  square  difference  patterns  are  being 
examined  to  determine  whether,  for  example,  the  set  of  Ss  in  Quadrant  C 
of  Table  25  showed  rapid  reversals  of  alpha  activity  briefly  after 
stimulus  triggering  so  that  they  might  actually  have  been  in  an  opposite 
cop..itive  mode  than  that  suggested  by  the  trigger  mode.  Another  inter- 
pretation is  that  in  some  Ss,  EEG  arousal  represents  a "busy"  mode 
whereby  external  stimuli  go  unattended. 

Experiment  3 - Asymmetry  of  Contingent  Negative  Variation 

In  using  measures  of  alpha  power  and  frequency  to  assess  cerebral 
dominance,  it  is  assumed  that  these  measures  reflect  states  of  cortical 
arousal--greater  arousal  being  associated  with  higher  frequency  and  less 
alpha  power.  Very  low-frequency  domains  in  the  EEG--D.C.  or  slow  poten- 
tial (SP)  changes--also  appear  to  reflect  the  state  of  cortical  excita- 
bility (Rebert  et  al.,  1967;  Rebert,  1972),  and  such  changes  often 
reflect  alteration  of  the  cortical  state  better  than  does  the  oscillatory 
EEG  (Rebert,  1973).  McAdam  and  Whittaker  (1971)  have  provided  data 
suggesting  that  slow  motor  readiness  potentials  are  asymmetrically 
distributed  in  the  hemispheres  when  Ss  prepare  to  make  meaningful  language 
sounds  in  contrast  to  nonsense  sounds,  and  Low  et  al.  (1974)  have  re- 
ported that  the  CNV  is  larger  in  the  language  hemisphere  when  a S vocal- 
izes a word  in  response  to  an  imperative  stimulus.  The  McAdam-Wh it taker 
study  has  been  sharply  criticized  on  methodological  grounds  (Morrell  and 
Huntington,  1971;  Grabow  and  Elliot,  1974),  and  Low's  studies  confound 
motor  preparation  with  receptive  cognitive  activity.  Other  investigators 
attempting  to  study  CNV  asymmetry  and  cognitive  Differences  in  the  hemi- 
spheres (Marsh  and  Thompson,  1973)  have  not  been  successful.  A paradigm 
suggested  by  the  work  of  Costell  et  al.  (1972)  was  used  to  promote  the 
occurrence  of  CNV  asymmetry  related  to  cognitive  differences  of  the 
hemispheres.  Costell  et  al.  used  the  same  stimulus  as  and  S2,  but 
presented  it  as  Sj  only  briefly  enough  to  give  j5s  a clue  as  to  its  con- 
tents (nude  figure),  and  then  with  longer  exposure  as  S2.  Similarly,  in 
the  experiment  described  here,  either  words  or  spatial  patterns  were 
used  a?  both  WS  and  IS.  The  IS  wis,  however,  either  an  antonym  or  synonym 
of  the  fi’st  word  or  the  same  or  different  pattern  as  the  WS.  Tlus,  the 
appearanc  i of  a word  WS  should-  have  promoted  relatively  greater  left 
hemisphere  arousal  and  vice  versa  for  patterns.  Verbal  and  nonverbal 
trials  were  intermixed  randomly. 
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Recordings  were  made  from  the  parietal  lobes  near  the  angular  gyrus 
(.an  area  related  to  reading  on  the  left)  and  from  t'7  and  F8  of  the 
International  10-20  system,  near  Broca's  motor  speech  area  on  the  left 
and  its  right  homolog.  Vertex  recordings  also  were  made.  Reference  was 
linked  mastoids.  Vertical  EOGs  were  recorded  bipolarly  between  the 
superior  and  inferior  orbital  ridges.  Beckman  sintered  Ag-AgCl  electrodes 
were  used.  Potentials  were  amplified  with  Grass  Model  5P-1  chopper- 
stabilized  amplifiers  with  matched  time  constants  of  approximately  4.5 
sec.  The  interstimulus  interval  was  1.5  sec,  so  little  CNV  attenuation 
from  the  time  constant  would  be  expected.  Even  though  some  attenuation 
might  occur  because  of  the  TC , it  could  not  occur  in  a pattern  related 
to  experimental  manipulation.  That  the  TC  was  adequate  to  record  CNVs 
is  indicated  by  the  examples  shown  in  Figure  37. 

Figure  37  also  shows  differences  in  right  and  left  parietal  CNVs  in 
one  S during  trials  involving  words.  The  left  area  exhibits  a larger 
CNV  than  the  right.  Across  S s , such  differences  were  not  consistent  in 
the  parietal  region,  although  there  was  a mean  trend  in  that  direction. 

The  upper  curves  in  Figure  38  show  right  and  left  parietal  average  CNV 
amplitudes  on  word  and  pattern  trials.  There  is  also  a tendency  for 
right  CNVs  to  be  larger  than  left  during  pattern  trials,  but  the  Stimulus 
X Hemisphere  interaction  was  not  significant, 

Fronto-temporal  CNVs  are  shown  in  the  lower  curves  of  Figure  38. 

The  Stimulus  X Hemisphere  interaction  is  much  clearer  there  and  is  sig- 
nificant (F  = 6.93,  df  = 1.9,  p < 0.05).  CNV  amplitudes  at  the  vertex 
are  shown  in  the  upper  renter  of  the  graph.  These  data  reflect  a typical 
distribution  of  CNV  an.olitu Je--largest  at  the  vertex,  but  well  sized 
parietally  and  very  smail  f ronto-temporally . At  the  vertex  and  parietal 
placements,  CNVs  were  slightly  larger  to  fcords  than  patterns,  reflecting 
perhaps  greater  interest-evoking  capacity  cf  the  words  or  greater  diffi- 
culty and  more  distractive  disruption  of  the  CNVs  to  patterns. 

These  data  suggest  that  the  use  of  warning  stimuli  that  themselves 
require  processing  directly  related  to  hemispheric  specialization  leads 
to  a clearer  induction  of  differential  "sets”  and  CNVs  than  when  second- 
arily meaningful  cues  such  as  different  tones  or  lights  are  used  as 

warning  signs  s to  induce  differencial  sets  (e.g. , Marsh  and  Thompson, 
1973).  ’ 

One  possible  criticism  of  these  results  became  evident  on  study  of 
recent  accounts  in  the  literature  concerning  eye  movements  related  to 
cognitive  activity.  Several  papers  (e.g.,  Ornstein,  1973)  indicate  that 
lateral  eye  movements  are  made  in  a direction  opposite  an  activated 
hemisphere--e.g. , right  movements  with  left  hemisphere  arousal.  Because 
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FIGURE  37 


XAMF'LE  FROM  ONE  SUBJECT  OF  AVERAGE  CNVs  (n  = 12/average)  FROM 
EFT  AND  RIGHT  PARIETAL  AREAS  DURING  PERFORMANCE  IN  A WORD- 
kNTICIPATION  AND  -DISCRIMINATION  TASK  THAT  WAS  INTERMIXED 
i/ITH  A SPATIAL  TASK 
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ACROSS  SUBJECTS  IN  PARIETAL 
AND  FRONTAL  LEADS  DURING 
ANTICIPATION  OF  WORDS  OR 
PATTERNS 


of  channel  limitations,  vertical  but  not  horizontal  EOGs  were  measured 
in  these  Ss , rrimarily  because  lateral  eye  movements  typically  do  not 
occur  in  the  CNV  paradigm.  Donchin  (personal  communication)  did  not 
detect  lateral  eye  movements  in  j3s  engaged  in  a cognitive-CNV  experiment 
similar  to  the  one  reported  here,  and  a second  test  of  one  of  our  S!s 
revealed  uo  lateral  eye  movements  as  measured  by  lateral  EOGs  (between 
the  two  outer  canthi). 

On  slow  RT  trials  to  words,  mean  CNV  amplitude  was  slightly  larger 
(0.7  uV)  in  the  right  than  left  parietal  area  but  was  larger  in  the 
left  (1.2  uV)  than  the  right  on  fast  RT  trials.  On  pattern  trials,  CNV 
was  the  same  in  both  hemispheres  (parietally)  when  RT  was  fast  but  was 
larger  on  the  left  when  RT  was  slow.  Although  not  statistically  reliable, 
these  trends  suggest  that  performance  is  worse  when  interhemispher ic 
arousal  is  patterned  contrary  to  the  cognitive  demands  of  the  task. 
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